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ABSTRACT 

This  thesis  proposes  and  validates  an  interrogative 
diagnostic  model  of  computer-aided,  adaptive  testing.  The 
model  uses  an  algorithm  based  on  Bayesian  techniques  to 
determine  both  the  number  and  the  difficulty  level  of 
questions  to  present.  The  power  of  the  model  lies  in  its 
ability  to  automatically  adapt  a  testing  session  to  the 
individual's  level  of  knowledge  by  choosing  the  domain,  type 
and  number  of  questions.  The  model  was  implemented  as  a 
computer  program  and  was  tested  in  an  experiment  with  34 
military  officers  who  were  masters  degree  students.  Analysis 
of  the  results  indicated  that  the  IDM  can  evaluate  student 
knowledge  as  effectively  as  traditional  written  examinations 
but  with  a  significantly  less  number  of  questions  and 
shorter  test  duration. 
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I .   INTRODUCTION 

In  education  it  is  crucial  to  access  objectively  and 
accurately  a  student's  level  of  proficiency.  Computer-Aided 
Testing  (CAT)  is  becoming  one  of  the  most  effective 
evaluation  tools  available  to  the  educator.  The  greatest 
challenge  to  developing  computer-aided  tests  is  modeling  a 
testing  session  in  an  intelligent  manner. 

Most  current  computer-aided  tests  are  based  on 
enumerative  or  adaptive  methods  [Bejar  1986,  Thomas  1986, 
Larson  1987].  Enumerative  testing  is  similar  to  traditional 
written  examinations  in  which  an  individual  is  asked  a  fixed 
number  of  predetermined  questions.  Adaptive  testing  on  the 
other  hand  selects  both  the  number  and  type  of  questions 
presented  to  the  individual  according  to  his  personal  skills 
and  previously  aquired  knowledge. 

This  thesis  proposes  and  validates  a  computer-aided 
testing  model  called  the  Interrogative  Diagnostic  Model 
(IDM).  The  model  is  both  declarative  and  adaptive  in 
nature.  In  other  words,  a  pool  of  questions  are  explicitly 
stored  in  a  test  bank  along  with  their  difficulty  level. 
The  model  uses  an  algorithm  based  on  Bayesian  techniques  to 
determine  both  the  number  and  the  difficulty  level  of 
questions  to  present.  The  power  of  the  model  lies  in  its 
ability  to   automatically  adapt  a  testing   session   to   the 


individual's  level  of  knowledge  by  choosing  the  domain,  type 
and  number  of  questions. 

The  model  was  implemented  as  a  computer  program  and  was 
validated  in  an  experiment  with  34  subjects.  Analysis  of  the 
results  indicated  that  the  IDM  can  evaluate  student 
knowledge  as  effectively  as  traditional  written  examinations 
but  with  a  significantly  less  number  of  questions  and 
shorter  test  duration.  Chapter  II  of  the  paper  reviews 
related  literature  in  the  area  of  computer  aided  adaptive 
testing.  Chapter  III  desribes  the  IDM  in  detail.  The 
subsequent  chapter  discusses  the  computer  implementation  of 
the  model.  Design  and  administration  of  the  experiment  is 
described  in  Chapter  V.  The  statistical  analysis  of  the 
results  are  presented  in  Chapter  VI.  Chapter  VII  concludes 
by  stating  the  advantages  and  disadvantages  of  the  model  and 
suggesting  directions  for  future. 


II  .  LITERATURE  REVIEW 

Although  considerable  work  has  been  done  in  the  area  of 
computer-aided  instruction,  research  in  computer-aided 
adaptive  testing  has  been  sparse  [Brown  1977,  Clancey  1984, 
Clancey  1986,  Davis  1983].  Weiss  [Weiss  1977]  described  a 
number  of  strategies  for  adapting  test  items  to  individuals 
differing  abilities.  Tennyson  and  Ruthen  [Tennyson  1977] 
verified  that  an  adaptive  strategy  in  selecting  the  number 
of  instances  needed  to  learn  concepts  was  more  efficient 
than  either  partially  adaptive  or  non-adaptive  strategies. 
Further  evidence  that  adaptive  testing  has  validity  was 
presented  in  an  experiment  by  Thathon  and  Kruawan  [Thathon 
1985]  in  which  a  self  scoring,  flexi-level  adaptive  testing 
procedure  proved  both  effective  and  practical. 

Bayesian  statistical  techniques  have  also  been 
investigated  in  the  adaptive  testing.  Sympson  [Sympson 
1978]  applied  Bayesian  techniques  to  measure  ability  of 
candidates  using  a  hypothetical  20  item  adaptive  test. 
Bayesian  analysis  has  also  been  applied  to  illustrate  the 
basic  mathematical  concepts  involved  in  scoring  multiple 
choice  achievement/aptitude  tests  [Warm  1978,  Vale  1981]. 
Further  application  of  Bayesian  analysis  to  adaptive  testing 
was  done  by  Sivasankaran  and  Bui  [ Sivasankaran  1986].  Their 
work  provides  a  basis  for  the  IDM. 


Methods  of  interpeting  a  candidate's  response  and 
immediate  modification  of  the  test  itself  were  discussed  by 
McArthur  and  Chou  [McArthur  1984].  Loyd  [Loyd  1984] 
presented  evidence  that  a  two-stage  adaptive  testing 
procedure,  in  which  the  first  stage  routes  the  individual  to 
second  stage  which  focuses  on  the  individuals  ability  level 
is  a  more  precise  measure  than  a  single  test.  Bejar  [Bejar 
1986]  describes  a  further  development  in  adaptive  testing  in 
which  the  testing  program  not  only  decides  the  number  of 
questions  but  also  actually  generates  new  questions  based  on 
the  ability  of  the  person  being  tested. 

However  no  attempt  has  been  made  to  investigate  the 
effects  of  incorporating  learning  theory  into  computer-aided 
adaptive  testing.  The  interrogative  diagnostic  model 
presented  in  this  paper  makes  use  of  Bayesian  techniques  and 
incorporates  a  multi-stage  testing  procedure  based  on 
cognitive  learning  theory  proposed  by  Kibler  et  al  [Kibler 
1970].  The  experiment  conducted  using  the  IDM  was  patterned 
after  the  design  of  Thathon  and  Kruawan  [Thathon  1985]  in 
that  a  group  of  individuals  was  given  both  a  written  and  a 
computer  test  covering  the  same  topics  and  the  results  of 
the  exams  compared. 


III.  THE  INTERROGATIVE  DIAGNOSTIC  HOJ2EL. 

A.   INTERROGATIVE  LEARNING  MEASUREMENT 

According  to  the  learning  theory  proposed  by  Kibler  et 
al.  [Kibler  1970],  an  individual  progresses  through 
different  levels  of  learning  (Table  1).  Levels  1  and  2  form 
the  first  phase  in  which  the  individual  begins  to  recall  or 
recognize  information  and  be  able  to  translate  it  from  one 
form  to  another.  These  two  levels  draw  heavily  on  the 
ability  to  memorize  information.  During  the  second  phase  of 
the  learning  process,  levels  3  through  6,  the  individual 
assimilates  and  integrates  information  of  increasing 
complexity. 

The  IDM  has  accordingly  two  types  of  questions  which 
directly  correspond  to  the  these  two  phases  of  learning. 
These  are  the  Supplementary  and  Critical  questions. 
Supplementary  questions  are  those  questions  which  are  basic 
in  nature  and  cover  levels  1  &  2.  Critical  questions  are 
those  questions  which  are  advanced  in  nature  and  cover 
levels  3-6. 

The  rationale  for  the  Interrogative  Diagnostic  Model 
(IDM)  of  testing  is  based  on  the  dynamics  of  an  oral 
examination.  Before  an  oral  examination,  the  examiner 
prepares  a  set  of  topics  to  be  covered  and  within  them 
identifies  specific  questions.  But  the  difficulty  level   and 


number  of  questions  actually  selected  and  presented  during 
the  examination  are  guided  by  the  student's  answers  to 
previous  questions.  This  type  of  interaction  can  best  be 
modeled  by  incorporating  Bayesian  techniques.  The  IDM  makes 
use  of  these  techniques. 

TABLE  1   COGNITIVE  LEARNING  LEVELS  [Kibler  1970] 

LEVEL  (Lowest  -  1,  Highest  -  2) 

1  KNOWLEDGE--recall  or  recognize  information 

2  COMPREHENSION--translate  it  from  one  form  to  another 

3  APPLICATION--apply  a   known  procedure   to  a   new 

situation 

4  ANALYSIS--break  down  information 

5  SYNTHESIS--put  information  together 

6  EVALUATION--evaluate  something  according   to   some 

criterion 

B.   MODEL  EXECUTION  STEPS 

Figure  1  illustrates  the  model  in  the  form  of  a 
flowchart.  The  model  first  formulates  a  hypothesis  about  a 
student's  proficiency  in  a  given  topic  area  and  then 
evaluates  that  hypothesis  using  a  bank  of  questions.  The 
model  then  derives  conclusions  about  the  student's 
knowledge . 
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Figure  1   IDM  Flow  Chart 


The  IDM  consists  of  eight  steps: 

1.  A  topic  area  is  selected. 

2.  A  hypothesis  is  made  that  the  student  is  more  likely 
an  average  student  and  a  prior  probability  value 
initialized . 

3.  A  question  is  presented  to  the  student. 

4.  The  student  response  is  evaluated  and  the  earlier 
probability  is  updated. 

5.  The  probability  is  compared  with  evaluation  criteria 
and  further  questions  are  asked  if  the  criteria  are 
not  met. 

6.  The  hypothesis  is  accepted  or  rejected. 

a.  If  the  hypothesis  is  rejected  it  is  because  the 
student  is  either  above  or  below  average  and  proceeds 
to  step  8.  In  either  case  the  model  selects  a  new 
topic  area  in  which  to  query  the   student. 

b.  If  the  hypothesis  is  accepted  it  is  because  the 
student  is  average  and  the  model  proceeds  to  step  7. 

7.  A  decision  is  made  to: 

a.  end  the  test  if  all  topic  areas  have  been  covered. 

b.  select  supplemental  questions  if  the  critical 
questions  have  been  asked. 

c.  select   the   next   topic   if   the   supplemental 
questions  have  been  asked. 

8.  A  decision  is  made  to: 

a.  end  the  test  if  all  topic  areas  have  been  covered. 

b.  select  the  next  topic  area. 

The  Bayesian  theory  as  applied  in  the  interrogative 
diagnostic  model  steps  shown  above  is  presented  in  detail  in 
Appendix  A. 


C.   BAYESIAN  PROBABILITY  AND  QUESTION  ORDERING 

Since  Bayesian  probabilities  are  affected  by  the  ongoing 
performance  of  the  individual,  the  ordering  of  the  different 
questions  selected  during  the  testing  process  can 
significantly  affect  the  judgement  about  the  students 
proficiency.  Three  general  methods  of  question  ordering  are 
considered:  decreasing  difficulty,  increasing  difficulty  and 
alternating  sequence.  If  questions  are  presented  in  a 
decreasing  order  of  difficulty,  an  average  student 
encountering  the  more  difficult  questions  at  the  beginning 
of  a  testing  session  might  be  inappropriately  underevaluated 
since  he  would  reach  the  cutoff  threshold  probability  early. 
On  the  other  hand,  if  the  questions  are  presented  in  an 
increasing  order  of  difficulty,  the  average  student  might 
tend  to  be  judged  brighter  than  he  really  is.  A  third 
method  is  to  order  questions  in  an  alternating  sequence  of 
difficulty.  Theoretically,  several  other  methods  are  also 
possible . 

The  effect  of  the  three  orderings  on  the  student 
evaluation  is  pictorially  in  Figure  2.  The  first  graph 
shows  the  effect  of  a  decreasing  order  of  difficulty.  From 
this  graph  it  can  be  seen  that  the  moving  average  of  the 
difficulty  does  not  move  down  as  rapidly  as  the  actual 
difficulty.  Regions  I,  II,  and  III  of  the  graph  represent 
the  level  of  difficulty  that  a  poor,  average  and  superior 
student  could  reasonably  be  expected  to  handle  respectively. 
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Figure  2  Effects  of  Different  Question  Ordering 
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In  the  first  graph,  the  average  difficulty  level  does 
not  fall  into  the  average  region  until  the  fourth  question 
has  been  asked.  By  this  time  the  Bayesian  calculations  may 
have  already  concluded  that  the  student  is  poor  when  in 
reality  he  is  average. 

In  a  similar  fashion  the  second  graph,  shoving  an 
increasing  order  in  difficulty  level  might  overrate  an 
average  student.  In  this  case  the  average  difficulty  level 
is  below  the  actual  level  and  only  when  the  fourth  question 
is  reached  does  the  average  difficulty  level  move  into  the 
region  where  the  average  student  would  be  challanged.  By 
the  fourth  question  the  Bayesian  calculations  may  have 
concluded  that  the  student  is  superior  when  he  could  be 
average . 

The  alternating  sequence  shown  in  the  third  graph 
overcomes  the  limitations  of  the  previous  orderings  by 
alternating  the  question  level.  As  can  be  seen  the  average 
difficulty  level  quickly  moves  into  the  average  area  and 
remains  there  with  ever  smaller  oscillations.  By  remaining 
in  the  average  area  there  is  less  likelihood  that  an  average 
student  will  be  under  or  over  evaluated. 
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IV.  IMPLEMENTATION  QSL   THE.  diagnostic  model 

Implementation  of  the  interogative  diagnostic  model  was 
divided  into  two  main  areas:  program  design  and  question 
design.  Program  design  was  closely  allied  with  the  first 
assumption  and  Bayesian  statistical  techniques  presented 
earlier.  The  area  of  question  design  corresponded  closely 
with  the  second  assumption  that  questions  can  be  written 
which  test  different  levels  of  the  cognative  domain. 

A.   PROGRAM  DESIGN 

The  design  of  a  program  to  implement  the  model  was  quite 
straight  forward.  dBASE  III  Plus  was  the  language  selected 
for  implementation  since  the  questions  were  easily  stored 
and  manipulated  in  a  database.  Because  the  specifications 
and  structure  of  the  program  were  not  fixed,  a  prototyping 
approach  was  used  in  designing  and  writing  the  program.  The 
final  program  consists  of  eight  modules  whose  data  flow 
diagram  is  shown  in  Figure  3. 

The  MAIN  module  retrieves  criteria  to  be  used  in  the 
Bayesian  calculations,  opens  data  files,  selects  topic 
areas,  and  calls  subordinate  modules.  The  INFO  module 
presents  the  test  instructions  and  gets  student 
identifcation  information.  The  ASK  module  presents 
questions   to  the  student  by  calling  the  DOIT  and  QUESTION 
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Figure  3   Program  Structure  Chart 
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modules  in  the  case  of  critical  questions  or  from  data  files 
in  the  case  of  supplemental  questions. 

The  ASK  module  also  forwards  student's  response  to  the 
CORRECT  module  for  updating  posterior  probabilities.  The 
FACTOR  module  calculates  factorials  for  the  CORRECT  module 
and  the  TALLY  module  writes  the  question  identication 
number,  student  answer  and  time  taken  to  a  data  file. 

The   program  evaluates  the  student  in  each   topic   area 

using   the  Bayesian  technique  presented  in  Appendix  A.    The 

numerical   scores   given  by  the  computer   program   for   each 

topic  range  as  follows: 

5  Superior 

4  Above  average 

3  Average 

2  Below  average 

1  Poor 

A   pictorial  representation  of  the  scoring  process  is   shown 

in  Figure  4.   The  student's  total  score  for  the  entire   exam 

is  an  average  of  the  scores  in  each  topic  area. 

Upon  completion  of  coding  the  program  was  compiled  using 

CLIPPER.    This   conversion  allowed  the  program   to   execute 

appreciably  faster  and   eliminated  the  the  necessity  for  the 

student  to  begin  the  exam  by  first  invoking  dBASE  III  Plus. 

B.   QUESTION  DESIGN 

In  designing  questions  consideration  was  given  to 
question  type,  content,  cognitive  level,  ordering,  and 
presentation.   The  types  of  questions  used  in   the   computer 
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upper  cutoff  reftchftd 
itotonllt  tup+rtor 


upper  cutoff  exceeded 
student  It  above  average 


KmVT  CUKCT 

not  exceeded 
ttudent  it  poor 


toww  cutoff  not  exceeded 
etudentlt  below  average 


Figure  4  Computer  Scoring  Process 
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exam  were  limited  to  multiple  choice  and  true/false.  Due  to 
the  inability  of  the  dBASE  III  Plus  in  processing  natural 
language  essay  type  questions  were  excluded.  The  content  of 
the  questions  came  from  the  textbook  instructors  manual, 
past  examinations  and  the  instructor.  The  questions  were 
catagorized  by  the  instructor  into  critical  and 
supplementary  as  discussed  in  Chapter  III.  However,  instead 
of  assigning  a  specific  difflculy  measure  to  each  question, 
each  question  within  a  catagory  was  assigned  one  of  three 
difficulty  levels,  with  Level  1  being  the  least  difficult 
and  Level  3  being  the  most  difficult.  The  text  of  both  the 
critical  and  supplemental  questions  used  in  the  program  are 
presented  in  Appendix  C.  The  alternating  sequence  of 
difficulty  discussed  in  Chapter  III  was  used  to  order  the 
questions  within  a  cognitive  level. 

Presentation  of  the  questions  to  the  subjects  was 
challenging.  Many  of  the  critical  questions  were  too  long 
to  fit  on  the  standard  80  x  25  character  screen.  This 
problem  was  solved  by  partitioning  each  critical  question 
into  two  program  files.  Dividing  the  critical  questions 
into  two  screens  meant  that  the  students  were  required  to 
page  back  and  forth  between  screens  to  view  a  single 
question. 
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V.  RESEARCH  DESIGN 

A.  SETTING 

A  micro-computer  lab  equipped  with  25  IBM  PCs  was  used 
for  administration  of  the  computer  exam.  A  diskette 
containing  the  compiled  version  of  the  computer  exam  was 
given  to  each  student.  A  week  prior  to  the  actual  exam  a 
demonstration  exam  was  given  to  the  student  in  the  same  lab 
in  order  to  let  the  student  become  familiar  with  the  testing 
program.  To  further  ensure  that  the  students  did  not 
encounter  any  technical  problems  a  facilitator  was  present 
during  the  examination. 

B.  SUBJECTS 

The  subjects  selected  were  masters  degree  management 
information  systems  students  enrolled  in  a  database 
management  course  at  the  Naval  Postgraduate  School.  The 
subjects  were  of  a  homogenius  background;  all  were  junior 
military  officers  in  their  third  quarter  in  the  computer 
technology  program  The  course  was  divided  into  two  sections 
of  21  and  13  individuals  respectively,  and  the  same 
instructor  taught  both  sections. 

C.  EXAM  DESIGN 

The  subject  area,  database  management,  was  divided  into 
three   topic   areas,  basic  concepts,  advanced   concepts   and 
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methodologies  (normalization),  and  issues  related  to 
management  of  databases.  Each  topic  area  had  5  critical  and 
20  supplementary  questions.  Both  the  written  and  computer 
final  examinations  consisted  of  the  same  75  questions  (15 
critical  and  60  supplemental)  and  both  had  a  two  hour  time 
limit.  The  written  examination  presented  all  75  questions 
to  the  subjects  and  the  questions  were  not  grouped  according 
to  topic  area  or  cognitive  level.  The  computer  exam 
presented  a  varying  number  of  questions  to  the  subject 
according  to  the  Bayesian  evaluation  of  the  subject's 
previous  answers.  In  the  computer  exam  questions  were 
grouped  according  to  topic  area.  Upon  finishing  the  exam 
each  student  was  given  a  questionnaire  to  determine  his 
reaction  to  the  exam. 

In  order  to  compensate  for  any  learning  dynamics  which 
could  have  influenced  the  responses  from  the  first  test  to 
the  second  test,  the  experiment  was  conducted  in  the 
following  manner.  One  section  was  given  the  computer-aided 
exam  first,  followed  by  the  written  exam.  The  other  section 
was  given  the  written  exam  first  followed  by  the  computer 
exam. 

Upon  finishing  the  exam  each  student  was  given  a 
questionnaire  to  determine  his  reaction  to  the  exam  and  to 
attempt   to   determine  any  other  factors   which   might   have 
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influenced   his   score.   The  questionaire   is   presented   in 
Appendix  C. 

D.   HYPOTHESES 

Based  on  the  premise  that  recent  developments  in 
computer-aided  testing  tools  would  effectively  evaluate 
students,  the  goal  of  the  experiment  was  to  focus  on 
determining  how  such  tools  compare  with  traditional  written 
examinations.  More  specifically,  this  experiment  was 
designed  to  evaluate  the  validity  of  the  Interrogative 
Diagnostic  Model.   The  following  hypotheses  were  tested: 


HI:  The  scores  achieved  by  a  student  will  be  the  same 
irrespective  of  whether  the  student  takes  the  written 
or  computer  tests. 

H2:  The  evaluation  quality  both  of  the  computer  and  of 
the  written  exam  will  be  the  same. 

H3:  The  number  of  questions  presented  to  the  student  in 
the  computer  exam  will  be  the  same  as  the  number  of 
questions  in  the  written  exam. 

H4:  The  time  taken  to  complete  the  computer  exam  will  be 
the  same  as  the  time  taken  to  complete  the  written 
exam. 


E.   HYPOTHESIS  TESTING  METHODOLOGY 

In  testing  the  above  hypotheses,  a  factor  analysis  using 
the  student's  computer  score,  section  and  responses  to  the 
questionaire  was  performed  to  identify  dominant  factors.  A 
regression  analysis  was  done  with  these  factors  as 
independent  variables  against  the  written  exam  score  as   the 
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dependent  variable.  In  testing  HI,  the  significance  level  of 
the  coefficient  of  the  computer  score  term  was  used.  In 
addition,  the  written  exam  scores  were  converted  to  the 
computer  scale  and  a  difference  in  means  test  between  the 
computer  and  written  exam  scores  was  conducted. 

The  evaluation  quality  (H2)  for  both  the  written  final 
exam  and  the  computer  final  exam  was  be  measured  by 
comparing  these  scores  with  the  average  score  of  the 
student's  two  previous  written  examinations.  This  was 
considered  an  appropriate  measure  because  the  entire  course 
consisted  of  three  examinations  with  the  third  and  final 
examination  being  comprehensive  having  only  about  10  percent 
of  new  material. 

Since  the  computer  examination  could  theoretically 
present  the  same  number  of  questions  as  the  written 
examination,  it  was  expected  that  the  number  of  questions 
presented  and  time  taken  for  the  computer  exam  would  be  the 
same  as  those  of  the  written  exam.  H3  was  tested  by  counting 
the  number  of  questions  in  the  computer  and  written 
examinations  and  the  two  counts  were  compared  using  a 
dlffenence  in  means  test. 

In  testing  H4,  the  times  taken  by  the  student  for  the 
computer  exam  and  for  the  written  were  measured  and  compared 
using  a  difference  in  means  test. 
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VI.   ANALYSIS  OF  EXPERIMENT  RESULTS 

The  computer  program  automatically  saved  the  exam 
responses,  the  computed  score  and  the  time  taken  by  each 
student.  In  performing  the  factor  analysis,  eight  variables 
were  included  to  capture  each  of  the  following:  1)  the 
computer  score  (COMPSCOR),  2)  the  section  (SECTION), 
3) Inability  to  explain  answers  (XPLNANS),  4)  lack  of  essay 
questions  (NOESSYQ),  5)  comfort  level  in  using  computers 
(COMFLVL),  6)  percieved  effectiveness  (PCVDEFCT),  7) 
perceived  pressure  (PCVDPRES),  8)  general  feelings  toward 
computer  testing  (GENLATT). 

The  factor  extraction  technique  of  principle  axis 
factoring  with  varimax  rotation  was  used.  Items  which 
failed  to  demonstrate  convergent  and  discriminate  ability  by 
loading  cleanly  on  a  factor  were  deleted.  The  results  of 
this  analysis  are  reported  in  Table  A.  The  items  loaded  on 
four  factors.  They  were  the  computer  score,  the  section, 
perceived  effectiveness,  and  general  discomfort  with  the 
concept  of  computer  testing. 
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TABLE  2.  VARIMAX  ROTATED  FACTOR  MATRIX 


Variable/Factor 


SECTION 

-0.32772 

0.20560 

0.85924 

0.09138 

COMPSCOR 

0.05631 

0.87267 

0.21429 

-0.12195 

GENLATT 

0.88687 

0.04518 

-0.12319 

0.04472 

XPLNANS 

0.56898 

-0.38634 

0.15509 

-0.46568 

MOESSYQ 

0.65517 

-0.07445 

0.63430 

0.06525 

PCVDPRES 

0.79692 

0.26031 

-0.18654 

0.11812 

COMFLVL 

0.09884 

0.66550 

-0.04637 

0.35176 

PCVDEFCT 

0.10792 

0.04534 

0.12618 

0.91199 

Considering  only  variables  with  loads  above  0.80,  it 
can  be  seen  that  factor  1  loads  on  general  feelings  towards 
computer  testing,  factor  2  on  the  computer  score,  factor  3 
the  section  and  factor  4  on  the  perceived  effectiveness  of 
computer-aided  exams. 

Using  the  scores  of  these  four  factors  as  independent 
variables,  a  multiple  regression  was  run  against  the  the 
written  exam  scores.  The  results  of  this  regression  are 
shown  in  Table  3. 


TABLE   3a.  REGRESSION  OF  THE  FACTORS  AGAINST 

WRITTEN  EXAM  SCORE 


Independent  coefficient  std.  error  t-value   sig. level 
variable 

CONSTANT  60.766667  1.359042  44.7129  0.0000 

FACTOR1  0.868266  1.382275  0.6281  0.5356 

FACTOR2  7.751186  1.382275  5.6076  0.0000 

FACTOR3  1.52467  1.382275  1.1030  0.2805 

FACTOR4  -1.42072  1.382275  -1.0275  0.3140 

R-SQ.  (ADJ.)  =  0.5094  SE=7. 443780 

MAE  =   5.356040  DurbWat=  1.959 
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TABLE  3b.  ANALYSIS  OF  VARIANCE  FOR  THE  FULL  REGRESSION 


Source    Sum  of  Squares   DF  Mean  Square   F-Ratio   P-value 

Model  890.12     4     472.530     8.52791    .0002 

Error  1385.25    25      55.4099 

Total  (Cor.)     3275.37    29 

R-squared  =  0.577071 
Stnd.  error  of  est.  =  7.44378 
R-squared  (Adj.  for  d.f.)  =  0.509403 
Durbin-Watson  statistic  =  1.95859 


From   the   table  it  is  evident  that  factor   2   (computer 

score)  is  the  only  significant  factor.  A  separate  regression 

was   therefore   run   using  the  computer  score   as   the   only 

independent  variable.   Results  of  this  regression  are   shown 

in  Table  4. 

TABLE  4a.  REGRESSION  OF  COMPUTER  SCORE  AGAINST 

WRITTEN  EXAM  SCORE 


Independent   coefficient  std.  error    t-value   sig. level 
variable 

CONSTANT        32.075889  3.67456        8.7292       0.0000 

COMPSCOR        10.172458  1.251616       8.1275       0.0000 

R-SQ.  (ADJ.)  =  0.6635  SE  =   6.025962 

MAE  =  4.970321  DurbWat  =   2.076 


TABLE  4b.  ANALYSIS  OF  VARIENCE  FOR  THE  FULL  REGRESSION 


Source   Sum  of  Squares   DF  Mean  Square   F-Ratio   P-value 

Model  2398.63     1    2398.63       66.0556      .0000 

Error  1161.99    32      36.3122 

Total  (Corr.)   3560.62    33 

R-squared  =  0.673655    R-squared  (Adj.  for  d.f.)  =  0.663456 
Stnd.  error  of  est.  =  6.02596 
Durbin-Watson  statistic  =  2.07571 
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The  regression  was  rerun  after  transforming  the   written 

and  the  computer  scores  to  a  scale  with  origin  (0,0)  so  that 

the  constant  term  could  theoritically  be  reduced   to  zero. 

For   this  smallest  of  the  written  and  computer   scores   were 

subtracted   from  the   original  data.   The   effect   of   this 

transformation  on  the  regression  are  shown  in  Table  5. 

TABLE  5.  REGRESSION  OF  COMPUTER  SCORE  AGAINST 
WRITTEN  EXAM  SCORE  AFTER  DATA 
TRANSFORMATION 

Independent   coefficient   std.  error    t-value   sig. level 
variable 

CONSTANT        4.248347      2.498387       1.7004       0.0987 
COMPSCOR0      10.172458      1.251616       8.1275       0.0000 

R-SQ.  (ADJ.)  =  0.6635  SE  =  6.025962 

MAE  =  4.970321  DurbWat  =   2.076 

The   computed  t-value  for  the  constant  term   is   1.7004. 

Since   this   is  less  than  the  critical  value  at   95   percent 

confidence  level  (1.960),  the  constant  is  statistically  not 

significant.   Thus   it   was   considered  appropriate   to   to 

disregard  the  constant  term  and  to  force  the  the   regression 

line   through   (0,0).   The  results  of   this   regression  are 

presented  in  Table  6. 
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TABLE  6a.  REGRESSION  OF  COMPUTER  SCORE  AGAINST  WRITTEN 
EXAM  SCORE  AFTER  DATA  TRANSFORMATION  AND 
DISREGARDING  CONSTANT  TERM 


Independent   coefficient  std.  error    t-value   slg. level 

COMPSCORO       12.110138     0.532355      22.7483       0.0000 

R-SQ.  (ADJ.)  =  0.9401  SE  ■  6.196253 

MAE  =  5.002530  DurbWat  =   1.994 

TABLE  6b.  ANALYSIS  OF  VARIENCE  FOR  THE  FULL  REGRESSION 

Source    Sum  of  Squares   DF  Mean  Square   F-Ratio   P-value 

517.483      .0000 


Model 

19868.0 

1 

19868.0 

Error 

1266.99 

33 

38.39 

Total 

21135.0 

34 

R-squared  =  0.940053 
Stnd.  error  of  est.  =  6.19625 
R-squared  (Adj.  for  d.f.)  =  0.940053 
Durbin-Watson  statistic  =  1.99362 


The  the  high  values  of  R-squared,  the  F-ratio  and  the 
low  mean  square  error  tend  to  confirm  that  it  is  sound  to 
model  the  computer  score  as  an  important  factor  influencing 
the  written  examination  score.  The  difference  in  means  test 
was  also  performed  using  the  transformed  data  (Table  7).  It 
can  be  seen  that  Hi  cannot  be  rejected. 
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TABLE  7.  TWO-SAMPLE  ANALYSIS  RESULTS 


Sample  1 

Sample  2 

Pooled 

Sample 

Number  of  Obs . 

34 

34 

68 

Statistics: 

Average 

2.99149 

2.90776 

2.94963 

Variance 

1.86803 

1.7982 

1.83311 

Std.  Deviation 

1.36676 

1.34097 

1.35393 

Median 

2.89474 

3.2 

3.11316 

Difference  between  Means  =  0.0837214 

Conf.  Interval  For  Diff.  in  Means:      95    Percent 

(Equal  Vars.)    Smpll  -  Smpl2  -0.572049  0.739492     66  D.F. 

(Unequal  Vars.)  Smpll  -  Smpl2  -0.572054  0.739496   66.0  D.F. 

Ratio  of  Variances  =  1.03884 

Conf.  Interval  for  Ratio  of  Variances:   0     Percent 

Sample  1  v  Sample  2 

Hypothesis  Test  for  HO: 

Diff  =  0        Computed  t  statistic  =  0.254956 
vs  Alt:  NE        Sig.  Level  =  0.799549 
at  Alpha  =  0.05     so  do  not  reject  HO. 


The  second  hypothesis,  that  the  computer  exam  would 
exhibit  the  same  quality  of  evaluation  as  the  written  exam, 
was  tested  by  comparing  both  the  written  exam  scores  and  the 
computer  exam  scores  with  the  average  of  the  scores  of  two 
previous  exams  the  students  had  taken.  The  rationale  for 
this  comparison  was  discussed  previously  in  hypothesis 
testing  methodology.  The  combined  results  of  the  two 
previous  exams,  called  STUQUAL,  served  as  the  standard  in 
determining  quality.  When  the  the  written  and  computer 
scores  were  compared  with  STUQUAL,  the  constant  term 
generated  was  not  statistically  significant  and  it  was 
considered  appropriate  to  disregard  the  constant  and  force 
the  regression  through  zero.   The  results  of  this  regression 
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are  shown  in  Table  8.  These  results  show  an  R-squared  o£ 
0.9809  for  the  written  final  exam  and  0.9380  for  the 
computer  exam.  Though  there  is  a  difference,  it  is 
considered  slight,  and  based  on  these  results,  H2  cannot  be 
rejected. 


TABLE  8a.    REGRESSION  OF  WRITTEN  FINAL  SCORE 

AGAINST  PRIOR  AVERAGE  SCORE 


Ind. variable   coefficient   std. error   t-value   sig. level 

FINAL  1.426209       0.03463     41.1837       0.0000 

R-SQ.  (ADJ.)  =  0.9809  SE  =  12.437046 

MAE  =  10.019345  DurbWat  =  1.828 

TABLE  8b.   ANALYSIS  OF  VARIANCE  FOR  THE  FULL  REGRESSION 


Source 

Sum  of  Squares 

DF 

Mean  Square 

F-Ratio 

P-value 

Model 
Error 
Total 

262353. 
5104.44 

1 
33 
34 

262353. 
154.680 
267457. 

1696.10 

.0000 

R-squared  =  0.980915 
R-squared  (Adj.  for  d.f.)  =  0.980915 
Stnd.  error  of  est.  =  12.437 
Durbin-Watson  statistic  =  1.82842 


TABLE   8c.   REGRESSION  OF  COMPUTER  SCORE 
AGAINST  PRIOR  AVERAGE  SCORE 

Ind. variable    coefficient   std.  error    t-value    sig. level 

COMPSCOR         29.259444     1.308931     22.3537       0.0000 

R-SQ.  (ADJ.)  =  0.9380  SE  =  22.407366 

MAE  ■  17.488222  DurbWat  ■   1.516 
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TABLE  8d.  ANALYSIS  OF  VARIANCE  FOR  THE  FULL  REGRESSION 
Source    Sum  of  Squares   DF  Mean  Square   F-Ratio   P-value 

.0000 


Model 

250888. 

1 

250888. 

499.687 

Error 

16569. 

33 

502.09 

Total 

267457. 

34 

R-squared  =  0.93805 
R-squared  (Adj.  for  d.f.)  =  0.93805 
Stnd.  error  of  est.  =  22.4074 
Durbin-Watson  statistic  =  1.51624 


Hypothesis  3,  that  number  of  questions  presented  to  the 
student  in  the  computer  exam  vill  be  the  same  as  the  number 
on  the  written,  was  rejected.  The  written  exam  consisted  of 
75  questions.  The  number  of  questions  asked  by  the  computer 
ranged  from  9  to  60.  The  average  number  of  questions  asked 
was  29.  A  difference  in  means  test  was  performed  comparing 
the  number  of  questions  asked  by  the  computer  exam  with  the 
number  of  questions  in  the  written  exam  (Table  9).  Results 
of  the  analysis  show  that  H3  is  rejected. 

The  fourth  hypothesis  to  be  tested  was  that  the  time 
taken  to  complete  the  computer  exam  would  be  the  same  as  the 
time  taken  to  complete  the  written  exam.  The  times  for  both 
groups  on  each  exam  are  shown  in  Table  10a.  Because  the 
questions  in  the  computer  exam  were  the  same  as  those  on  the 
written  exam,  Section  1  students  who  took  the  written  exam 
first,  had  seen  all  the  questions  the  computer  could 
theoretically  ask.  On  the  other  hand  Section  2  students, 
who  had   taken  the  computer  exam  first,  had  only  seen  a 
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TABLE   9.  DIFFERENCE  IN   MEANS   BETWEEN   NUMBER 
OF  COMPUTER  ASKED  QUESTIONS 
AND  WRITTEN  TEST 


NUMQUEST 
Sample  Number  of  Obs.        34 

Statistics:        Average  29.0588 

Variance  181.148 

Std.  Deviation       13.4591 
Median  26 

Confidence  Interval  for  Mean:    95    Percent 

Sample  1      24.3616  33.756     33  D.F. 

Confidence  Interval  for  Variance:       0      Percent 

Sample  1 


Hypothesis  Test  for  HO: 

Mean  =  75  Computed  t  statistic  =  -19.9033 

vs  Alt:  NE  Sig.  Level  =  0 

at  Alpha  =0.05  so  reject  HO. 


portion  of  the  questions  of  the  written  exam.  To  avoid  any 
learning/memory  effects  that  could  have  occurred  between  the 
two  exams,  the  written  exam  time  of  Section  1  was  compared 
with  the  computer  exam  time  of  Section  2.  A  difference  in 
means  test  was  run  on  these  times  and  the  results  are  shown 
in  Table  10b.  Based  on  the  results  of  this  test  H4  was 
rejected. 

A  summary  of   the   results   of   this   experiment   are 
presented  in  Table  11. 
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TABLE  10a.   AVERAGE  EXAM  TIMES  (minutes) 

Computer   Exam   Time     Written  Exam  Time 


Section  1 
Section  2 
Combined 


30 
53 
39 


112 

83 

101 


TABLE  10b.   DIFFERENCE  IN  MEANS  BETWEEN  COMPUTER 

AND  WRITTEN  EXAM  TIMES 


CTIME 

WTIME 

Pooled 

Sample 

Number  of  Obs. 

13 

21 

34 

Statistics : 

Average 

52.7692 

112.095 

89.4118 

Variance 

236.192 

179.89 

201.004 

Std.  Deviation 

15.3685 

13.4123 

14.1776 

Median 

53 

120 

103.5 

Difference  between  Means  =  -59.326 
Conf.  Interval  For  Diff.  in  Means: 
(Equal  Vars.)    Smpll  -  Smpl2   -69 
(Unequal  Vars.)  Smpll  -  Smpl2   -70 


95    Percent 
5199  -49.1322  32  D.F. 
0266  -48.6254  22.9  D.F 


Ratio  of  Variances  =  1.312983 
Conf.  Interval  for  Ratio  of  Variances 

Sample  1  v  Sample  2 


Percent 


Hypothesis  Test  for  HO: 
Diff  =  0 
vs  Alt:  NE 
at  Alpha  =  0.05 


Computed  t  statistic  =  -11.8573 
Sig.  Level  =  3.00426E-13 
so  reject  HO. 
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TABLE  11.   SUMMARY  OF  RESULTS 


HI:  The  scores  achieved  by  a  student  will  be  the  same 
irrespective  of  whether  the  student  takes  the  written 
or  computer  tests. 

NOT  REJECTED 


H2:  The  evaluation  quality  both  of  the  computer  and  of 
the  written  exam  will  be  the  same. 

NOT  REJECTED 

H3:   The   number  of  questions  presented  to  the   student   in 
the   computer   exam  will  be  the  same  as  the   number   of 
questions  in  the  written  exam. 

REJECTED 

H4:  The  time  taken  to  complete  the  computer  exam  will  be 
the  same  as  than  the  time  taken  to  complete  the 
written  exam. 

REJECTED 
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VI.  CONCLUSIONS 

This  thesis  presented  the  Interrogative  Diagnostic  Model 
and  an  experiment  designed  to  test  its  validity.  The  model 
provides  an  adaptive  mechanism  to  tailor  testing  sessions  to 
individual  students.  The  results  of  the  experiment  indicate 
that  the  IDM  is  a  reliable  model  in  evaluating  student 
performance.  The  results  also  show  that  the  IDM  effectively 
evaluates  students  in  less  time  and  with  less  questions  than 
a   written   test. 

The  major  benefits  in  using  such  a  testing  system  of  are 
more  efficient  use  of  students'  and  instructors'  time  and 
less  frustration  in  test  taking  for  both  strong  and  weak 
students.  With  this  type  of  testing  strong  students  are  not 
bored  by  having  to  answer  questions  which  are  beneath  their 
level  of  learning  and  hand  weak  students  are  not  frustrated 
by  having  to  answer  questions  beyond  their  capabilities. 

Further  validation  testing  will  be  needed  before 
widespread  use  is  made  of  this  model.  Possible  future  model 
testing  scenarios  would  be  testing  the  model  in  a  classroom 
situation  in  which  both  computer  tests  and  written  tests  are 
actively  used  to  grade  students.  The  effects  that  computer 
interface  has  on  student  performance  is  also  area  of  related 
research . 
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One  possible  area  in  which  examinations  based  on  the  IDM 
might  prove  beneficial  is  placement  testing  in  industry  and 
the  military.  The  IDM  might  also  prove  valuable  when 
integrated  with  computer-aided  adaptive  instruction. 
However  used,  the  model  can  ultimately  prove  to  be  an 
effective  substitute  for  conventional  diagnostic  methods 
currently  in  use. 
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APPENDIX  A 
BAYESIAN  APPROACH  TO  THE  INTERROGATIVE  DIAGNOSTIC  MODEL 


In  applying  Bayesian  analysis  to  the  steps  of  the 
Interrogative  Diagnostic  Model  presented  in  Section  3,  two 
assumptions  are  necessary  t Sivasankaran  1986.  The  initial 
hypotheses  of  step  2  are  derived  by  considering  these  two 
assumptions.  The  first  is  that  nothing  is  known  about  the 
subject  and  there  is  a  50-50  chance  that  the  subject  is 
average.  This  possiblity  reflects  complete  uncertianty  and 
becomes  the  uninformed  prior  probablity  pu  =  0.50.  The 
second  assumption  is  that  the  student  has  been  evaluated, 
either  by  a  human  instructor  or  historical  trends,  and  that 
a  better  estimate  on  the  subject  is  available.  This  better 
estimate  is  termed  pe .  The  degree  of  conviction  in  the 
uninformed  versus  the  estimated  assumptions  is  a:b  where  a 
and  b  are  weights.  Expressed  in  probability  notation  this 
becomes 

P(BU)  =  a/(a+b) 
P(Be)  =  b/(a+b) 

where  Bu  denotes  the  strength  of  belief  in  the  uninformed 
opinion  and  Be  the  strength  in  the  belief  in  the  estimated 
opinion.  These  strengths  form  the  initial  hypotheses  of 
step  2 . 
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Let  M  be  the  number  of  questions  asked  in  a  given  topic 
area.  The  number  of  questions  answered  correctly  is  C  and 
each  event  is  denoted  E.  The  mathematical  formulation  is 
shown  below. 


P(E/BU) 


Pu   (1  -  Pu) 


M-C 


|M]  PXC   (1  -  Px) 


Bayes   theorm  P(B/A)  =  P(B)  P(A/B)/[P(B)  P]  may   now   be 
applied  as  follows: 


P(BU/E) 


|M|   puC    (l-pu)M 


(a/a+b) 


M 


"A 


M-C 


M  \         C 


■J 


Pu      d-Pu>  +      (b/a+b)    II  px      (1-PX) 


M-C 


and, 


P(BX/E)    =    1    -   P(BU/E) 


where  P(BU/E)  and  P(BX/E)  represent  the  updated  hypotheses 
that  the  student  has  correctly  answered  C  out  of  M 
questions.  Updating  the  hypotheses  is  done  by  steps  3  and  4 
of  the  model.  Step  three  asks  the  student  a  question  and 
step  4  calculates  the  updated  hypotheses  based  on  the 
student's  response. 
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Given  the  updated  hypotheises  there  are  two  possible 
courses  of  action.  The  first  is  to  compare  the  updated 
value  of  P(BX/E)  with  upper  and  lower  cutoff  criteria.  The 
upper  cutoff  criteria,  alpha,  represents  the  minimum 
strength  of  belief  in  the  estimated  opinion  that  is  required 
to  conclude  that  the  students  knowledge  is  good.  If  P(BX/E) 
exceeds  alpha,  the  student  is  judged  knowledgeable  enough  in 
the  topic  area  being  evaluated  that  no  further  questions  are 
needed.  Conversely,  the  lower  cutoff  criteria,  beta, 
represents  the  minimum  strength  of  belief  in  the  estimated 
opinion  required  to  discern  weak  students.  If  P(BX/E)  does 
not  exceed  beta,  the  student  is  judged  unknowledgeable  in 
the  topic  area  being  evaluated  and  no  further  questions  in 
that  topic  area  are  needed. 

The  second  course  of  action  is  necessary  when  all  the 
questions  in  the  critical  level  have  been  asked  and  the 
updated  hypothesis  P(BX/E)  lies  between  alpha  and  beta.  In 
this  case  the  student  is  considered  to  have  an  average 
knowledge  of  the  subject  area  and  the  model  then  proceeds  to 
the  lower  level  of  supplementary  questions  in  the  same  topic 
area  or  if  all  the  supplemental  questions  have  been  asked, 
the  model  proceeds  to  a  new  topic  area. 

IDM  step  5  compares  P(BX/E),  the  revised  estimated 
opinion,  with  the  cutoff  values  and  if  neither  of  the  cutoff 
criteria  have  been  been  reached  the  model  returns  to  step  4. 
If   one   of  the  criteria  has  been  reached  or   there   are   no 
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questions  remaining  to  be  asked  in  the  in  the  cognitive 
level,  the  model  proceeds  to  step  6.  Step  6  accepts  or 
rejects  the  updated  hypothesis  based  on  the  cutoff  criteria. 
If  student  has  exceeded  the  upper  cutoff  or  has  not  exceeded 
the  lower  cutoff,  the  hypothesis  that  the  student  is  average 
is  rejected.  This  means  that  the  student  has  been  evaluated 
as  either  surerior  or  poor  and,  in  either  case,  the  model 
proceeds  to  a  new  topic  area  by  returning  to  step  one.  If 
the  hypothesis  is  accepted  step  7  will  ask  further 
supplementary  questions  to  further  define  the  students 
knowledge  level  or,  if  the  supplenentary  questions  have  been 
asked,  will  proceed  to  a  new  topic  area. 
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APPENDIX  B 
PROGRAM  LISTING 


This  appendix  contains  a  listing  to  the  dBASE  III  Plus 
program  files  which  implementated  the  IDM.  The  appendix 
begins  with  a  list  of  the  memory  variables  and  datafield 
names  used  in  the  program  files.  The  programs  IQ1P1.PRG  and 
IQ1P2.PRG  are  an  example  of  the  programming  method  used  to 
implement  the  two  screen  critical  questions  mentioned  in 
Chapter  IV.  The  program  files  using  this  method  are 
referred  to  as  QUESTION  in  the  program  structure  chart  of 
Figure  3. 

MEMORY  VARIABLES 
Variable   Representing 

aa         degree  of  conviction  in  uninformed  probability 
aalfa      threshold  for  beliving  subject  is  above  average 

bb  degree  of  conviction  in  expert  probability 

bbeta  threshold  for  beliving  subject  is  below  average 

f  value  of  combinations  returned  from  factor. prg 

i  counter  in  factor. prg 

k  temporary  storage  variable  for  (numasked  -  right) 

kf  temporary  storage  variable  for  k! 

moreq      flag  indicating  if  further  supplemental  questions 
are  needed 

nf         temporary  storage  variable  for  numasked! 
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numasked    number  of  questions  asked  thus  far  of  a  certian 
topic  and  type 

numcrit     number  of  critical  questions  of  a  given  type 

numsup     number  of  supplementary  questions  of  a  given  type 

PBUE       Bayesian  probability  of  the  uninformed  opinion 
being  correct  given  the  event 

PBXE       Bayesian  probability  of  the  estimated  opinion 
being  correct  given  the  event 

PEBU       probability  of  the  event  given  the  uninformed 
opinion 

PEBX       probability  of  the  event  given  the  estimated 
opinion 

rf         temporary  storage  variable  for  right! 

right      number  of  questions  answered  from  a  given  type 
and  topic  area 

SBU        strength  of  belief  in  uninformed  opinion 

SBE        strength  of  belief  in  estimated  opinion 

stop        flag  indicating  if  further  critical  questions 
are  needed 

temp       temporary  storage  variable  used  in  factor. prg 

wrong      number  of  questions  answered  incorrectly  from  a 
given  type  and  topic  area 

u  uninformed  prior  probability  that  a  subject 

is  not  average 

x  expert's  prior  probability  that  a  subject  is 

above  average 
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Field  Names 

A 

ALFA 

ANSWER 

AREA 

ASTART 

AFINISH 

B 

BETA 

CUTOFF 

NAME 

LEVEL 

PROGRAM1 

PROGRAM2 

PU 

PX 

QNUM 
QUESTION 


DATAFIELD  NAMES 

Remarks 

degree  of  conviction  in  uninformed  probability 

threshold  for  beliving  subject  is  above  average 

the  answer  to  a  particular  question 

the  topic  area  of  questions  being  asked 

the  time  the  subject  was  presented  a  question 

the  time  the  subject  answered  the  question 

the  degree  of  conviction  in  expert  probability 

threshold  for  beliving  subject  is  below  average 

minimum  percentage  of  supplemental  questions  the 
program  will  present  to  the  subject 

subjects  last  name 

difficulty  level  of  a  particular  question 

the  first  page  program  of  a  critical  question 

the  second  page  program  of  a  critical  question 

uninformed  prior  probability  that  a  subject 
is  not  average 

expert's  prior  probability  that  a  subject  is 
above  average 

question  identifier 

memo  entry  containing  the  question  and 
multiple  choice  answers 


SRESPONSE   the  subjects  answer  to  the  question 
SSN 


SUP 


TYPE 


last  four  digits  of  a  subject's  SSN  for 
identification  purposes 

name  of  the  DBF  file  containing  the  supplemental 
questions 

cognative  domain  level  of  question  being  asked 
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* 

Program. . 

:  MAIN.PRG 

* 

Author 

:  PAT  ODONNELL 

* 

Date 

SPRING  1988 

* 

Notice . . . 

Copyright   (c) 

* 

Reserved 

* 

•  THESIS 

1988,   PAT-SIV,   All    Rights 


SET  TALK  OFF 
SET  BELL  OFF 
SET  ESCAPE  ON 
SET  CONFIRM  ON 
SET  COLOR  TO  W+/B 

*  This  is  the  main  program  module  for  a  computer  aided 

*  testing  program.   This  module  calls  initilaizes   the 

*  criterion   values,  and  then  calls  two  other   modules 

*  which  do  the  work  of  the  program. 


aa  =  0 

bb  =  0 

aalfa  =  0 

bbeta  =  0 

areanum  = 

■  1 

X  =  0 

U  =  0 

ccutoff  = 

■  0 

USE  criterion 

STORE 

ALFA 

TO  aalfa 

STORE 

BETA 

TO  bbeta 

STORE 

A  TO 

aa 

STORE 

B  TO 

bb 

STORE 

PX  TO  x 

STORE 

PU  TO  u 

STORE 

CUTOFF  TO  ccutoff 

USE 

SELECT  1 

USE  blkboard 

SELECT  2 

USE  dbarea 

CLEAR 

TEXT 
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WELCOME  TO  THE 
COMPUTER  AIDED  TESTING  SECTION 
OF  THE  IS-4183  FINAL  EXAMINATION 


ENDTEXT 
8  22,10 
WAIT 

CLEAR 
DO  INFO 


STORE  "  "  TO  aarea 
SELECT  2 
GO  TOP 
DO  WHILE  areanum  <=  3 

STORE  area  TO  aarea 

DO  ASK 

SELECT  2 

areanum  =  areanum  +  1 

SKIP 
ENDDO 

CLEAR 
TEXT 

ENDTEXT 

*  EOF  MAIN.PRG 
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* 

Program. . 

:  INFO.PRG 

* 

Author . . . 

:  PAT  ODONNELL 

* 

SPRING  1988 

* 

Notice . . . 

:   Copyright   (c) 

* 

Reserved 

* 

■  THESIS 

1988,   PAT-SIV,   All    Rights 


SET  TALK  OFF 
SET  BELL  OFF 
SET  CONFIRM  ON 


*  This  program  module  introduces  the  program   to   the 

*  student, gives  instructions  on  using  the  program  and 

*  gets  the  student's  name  and  last  four  digits  of  his 

*  SSN  for  scoring. 

CLEAR 
TEXT 


Good  morning  and  welcome  to  an  experiment  in 
computer  aided  testing.  The  purpose  of  this  experiment 
is  to  try  and  determine  whether  a  computer,  using  a 
small  bank  of  questions  and  Bayesian  analysis,  can 
determine  an  individual's  knowledge  level  in  a  given 
subject  area  as  effectively  as  a  traditional  written 
exam.  The  program  presented  here  is  not  a  mere  test 
with  a  fixed  number  of  questions  to  be  answered.  This 
program  adjusts  both  the  number  and  type  of  questions 
to  the  individual.  Should  this  experiment  prove 
successful,  testing  programs  of  this  type  could  have 
broad  application  in  educational,  industrial  and 
military  environments. 

ENDTEXT 

622,10 

WAIT 

CLEAR 
TEXT 


INSTRUCTIONS:  You  will  be  shown  a  series  of  questions 
and  the  program  will  prompt  you  for  the  answer.  Just 
type  in  your  answer  followed  by  a  carriage  return.  Some 
questions  are  presented  in  two  screens.  You  may  page 
back  and  forth  between  screens  by  following  the   system 
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prompts.  If  you  have  any  questions  there  will  be 
someone  in  the  lab  to  answer  them. 

IMPORTANT:  Be  aware  that  you  must  answer  each  question 
in  order,  in  other  words  you  cannot  skip  a  question  and 
return  to  it  later. 

Professor   Sivasankaran   will   use   the   higher    score 
between  this  computer  aided  test  and  your  written  exam, 
so  this  does  "count." 
Good  luck ! 
ENDTEXT 
e  22,10 
WAIT 

CLEAR 

SELECT  1 

MNAME  =  '  ' 

MSSN  =  0000 

%      10,10  SAY  [Enter  your  lastname  and  last  four   digits 

of  your ] 
6  11,10  SAY  [social  security  number  in  the   appropriate 

blocks .  ] 
@  14,10  SAY  "  LASTNAME:  " 

%    14,3  3  GET  MNAME  PICTURE  "I!!!!!!!!  I  l!  !!!!!  I  !!  " 
READ 

8  16,10  SAY  "  SSN:  " 

@  16,29  GET  MSSN  PICTURE  "#»##" 
READ 

APPEND  BLANK 
REPLACE  NAME  WITH  MNAME,  SSN  WITH  MSSN 

RETURN 

*  EOF  INFO.PRG 
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* 

Program. . 

:  ASK.PRG 

* 

Author . . . 

:  PAT  ODONNELL 

* 

SPRING  1988 

* 

Notice 

Copyright  (c)  1988, 

* 

Reserved 

* 

THESIS 

PAT-SIV,   All   Rights 


SET  TALK  OFF 
SET  BELL  OFF 
SET  CONFIRM  ON 


*  This  program  asks  the  student  a  series  of   critical 

*  and  complementary  questions  from  the  topic  database 

*  file.    The   program  forwards  the  results   of   the 

*  questioning  to  calculation  modules   for   bayesian 

*  analysis. 

numsup  =  00 
numcrit  =  5 
numasked  =  00 
stop  =  'NO' 
right  =  00 
moreq  =  'NO' 
STORE  "  "  TO  supp 
STORE  "C"  TO  type 
STORE  '  '  TO  start 
STORE  '  '  TO  finish 

SELECT  3 

USE  &aarea 

GO  TOP 

STORE  sup  to  supp 

DO  WHILE  .NOT.  EOF ( )  .AND.  stop  =  'NO' 

STORE  programl  TO  pagel 
STORE  program2  TO  page2 
STORE  "  "  TO  response 

goback  =  "Y" 

DO  WHILE  goback  <>  "N" 

DO  DOIT 
ENDDO 

8  24,05  SAY  "Your  answer  to  question  " 

8  24,29  SAY  page2 

8  24,32  SAY  "  is:  [one  of  a,  b,  c  ...]" 

@  24,  59  get  response  picture  • ! ' 

READ 

§  24,05  SAY  "  You  chose  " 
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@  24,25  SAY  response 

S  24,26  SAY  "   Press  ENTER  to  confirm 

8  24,25  GET  response  PICTURE  » !' 

READ 

STORE  TIME()  TO  finish 

IF  response  =  answer 
right  =  right  +  1 
ENDIF 

SELECT  1 

APPEND  BLANK 

REPLACE  qnum  WITH  page 2 

REPLACE  sresponse  WITH  response 

REPLACE  astart  WITH  start 

REPLACE  afinish  WITH  finish 

SELECT  3 

numasked  =  nuroasked  +  1 

DO  correct 

SKIP 


ENDDO 


IF  moreq  =  'YES' 

SELECT  4 
USE  &supp 
GO  TOP 

DO  WHILE  .NOT.  EOF ( ) 

IF  type  ='S' 

numsup  =  numsup  +  1 

ENDIF 

SKIP 
ENDDO 

stop  =  'NO' 

numasked  =  0 

right  =  0 

STORE  •  ■  TO  suppnum 

GO  TOP 

DO  WHILE  .NOT.  EOF()  .AND.  stop  =  'NO' 

IF  type  =  '  S' 

STORE  STR(RECNO(),  2,  0)  TO  suppnum 
CLEAR 
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",  QUESTION 


STORE  TIME()  TO  start 
@  3,10 
?  M 
? 

STORE  •  '  TO  response 

@  22,20  SAY  "Your  answer  to  question  " 
£  22,44  SAY  supp 
@  22,45  SAY  "SQ     " 
§  22,47  SAY  suppnum 
8  22,49  SAY  "  is  M 
@  22,53  GET  response  PICTURE  "J" 
READ 


8  22,10  SAY  "  You  chose  M 

@  22,30  SAY  response 

8  22,31  SAY  "   Press  ENTER  to  confirm 
@  22,30  GET  response  PICTURE  ' !' 
READ 

STORE  TIME( )  TO  finish 

IF  response  =  answer 
right  =  right  +  1 
ENDIF 

SELECT  1 

APPEND  BLANK 

REPLACE  qnum  WITH  suppnum 

REPLACE  sresponse  WITH  response 

REPLACE  astart  WITH  start 

REPLACE  afinish  WITH  finish 

SELECT  4 

numasked  =  numasked  +  1 
DO  correct 
SELECT  4 


ENDIF 

SKIP 

ENDDO 

ENDIF 

RETURN 

* 

eof  ask.prg 
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*  Program. . i 

IQ1P1.PRG 

*  Author . . . : 

PAT  ODONNELL 

*  Date : 

SPRING  1988 

*  Notice . . . : 

Copyright   (c) 

*  Reserved 

THESIS 

CLEAR 

TEXT 

1988,   PAT-SIV,   All    Rights 


Consider   a  STUDENT  file  with  the  following  fields   used 
in  a  file  processing  system: 


Name 

character 

15 

Rank 

character 

6 

Address 

character 

20 

City 

character 

10 

State 

character 

2 

Zip 

numeric 

5 

Salary 

numeric 

5 

Age 

numeric 

2 

One  of  the  records  in  the  STUDENT  file  contains  the 
following  values: 

Smith  LCDR  202FranklinSt  Monterey  CA  93940  42000  33 

ENDTEXT 
3  24,10 
WAIT 
RETURN 

*EOF  IQ1P1.PRG 
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*  Program. 

*  Author . . 

*  Date. . . . 


IQ1P2.PRG 
PAT  ODONNELL 
SPRING  1988 

*  Notice...:   Copyright   (c)  1988,  PAT-SIV,   All   Rights 

*  Reserved 

*  Notes :  THESIS 

CLEAR 
TEXT 


If  the  Postal  authorities  decided  to  increase  the  zip 
code  length  to  six  digits  and  the  application  program  was 
so  modified,  John's  salary  (based  on  the  existing 
data  files)  would: 

a.  not  change  at  all 

b.  would  increase  by  $3 

c.  would  increase  by  $100,003 

d.  would  increase  by  $30,000 

e.  would  decrease  by  $21997 

f.  would  decrease  by  $40000 

g.  None  of  the  above  is  true 

ENDTEXT 
RETURN 

*EOF  IQ2P2.PRG 
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*  Program. . 

*  Author . . . 

*  Date 

*  Notice . . . 

*  Reserved 

*  Notes :  THESIS 


DOIT.PRG 
PAT  ODONNELL 
SPRING  1988 
Copyright   (c)   1988,   PAT-SIV,   All    Rights 


SET  TALK  OFF 
SET  BELL  OFF 
SET  CONFIRM  ON 

*  This   is   the  program  module   which  presents   the 

*  critical  questions  to  the  student. 


STORE  "  "  TO  goback 

STORE  TIME()  TO  start 

DO  &pagel 

DO  &page2 

@  24,5  SAY  "Do  you  want  to  go  back  to  the  previous   page? 

(Y  or  N)" 
§  24,59  GET  goback  picture  "!" 
READ 

RETURN 

*eof  DOIT.PRG 
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* 

Program. . . 

CORRECT. PRG 

* 

Author . . . i 

PAT  ODONNELL 

* 

SPRING  1988 

* 

Notice .  .  .  : 

Copyright   (c) 

* 

Reserved 

* 

THESIS 

1988,   PAT-SIV,   All    Rights 


SET  TALK  OFF 
SET  BELL  OFF 
SET  CONFIRM  ON 


This   program  module   calculates   the    Bayesian 
probabilities  of  the  student's  correct  answers. 


CLEAR 
f  =  0 
eval  =  0 

do  FACTOR  with  numasked,  right,  f 

SBU  =  aa/(aa  +  bb) 
SBX  =  bb/(aa  +  bb) 

PEBU  =  £  *  (u  **  right)  *  ((1  -  u)  **  (numasked  -  right)) 
PEBX  =  f  *  (x  **  right)  *  ((1  -  x)  **  (numasked  -  right)) 

PBUE  =  (SBU  *  PEBU)  /  ((SBU  *  PEBU)  +  (SBX  *  PEBX)) 

PBXE  =  1  -  PBUE 


*@14,10  say  "PBXE 
*@14,18  say  PBXE 
*$20,10 
*WAIT 

CLEAR 


ii 


DO 

CASE 

CASE  type  =  •C 

.AND. 

PBXE  >  =  aalfa 

stop  =  'YES' 

eval  =  1 

DO  tally 

CASE  type  =  'C 

.AND. 

PBXE  <=  bbeta 

stop  =  'YES' 

eval  =  5 

DO  tally 

CASE  type  =  'C 

.AND. 

numasked  =  numcrit 

moreq  =  'YES' 

CASE  type  =  'S' 

IF  (numasked/numsup)  >=  ccutoff  .AND.  PBXE  >=  aalfa 
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stop  =  'YES' 
eval  =  2 
DO  tally 
ENDIF 

IF  (numasked/numsup)  >=  ccutoff  .AND.  PBXE  <=  bbeta 

stop  =  'YES' 

eval  =  3 

DO  tally 
ENDIF 

CASE  type  =  ' S'  .AND.  numasked  =  numsup 
eval  =  4 
DO  tally 

ENDCASE 

RETURN 

eof  correct. prg 
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* 

Program. . 

•  TALLY. PRG 

* 

Author 

:  PAT  ODONNELL 

* 

'  SPRING  1988 

* 

Notice 

Copyright   (c) 

* 

Reserved 

* 

-  THESIS 

1988,   PAT-SIV,   All    Rights 


SET  TALK  OFF 
SET  BELL  OFF 
SET  CONFIRM  ON 

*  This  is  the  program  module  which  appends   the   area 

*  evaluations  to  the  blkboard  file. 


SELECT  1 

APPEND  BLANK 

REPLACE  name  WITH  aarea 

REPLACE  ssn  WITH  eval 

SELECT  3 

RETURN 

*  EOF  TALLY. PRG 
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*  Program. . 

*  Author . . . 

*  Date 

*  Notice . . . 

*  Reserved 

*  Notes :  THESIS 


FACTOR. PRG 
PAT  O'DONNELL 
SPRING  1988 
Copyright   (c)  1988,  PAT-SIV,   All   rights 


PARAMETERS  numasked,  right,  £ 

SET  TALK  OFF 
SET  BELL  OFF 

temp=l 
i  =  1 

do  while  i  <=  numasked 

temp  =  temp  *  i 

i  =  i  +  1 
enddo 
nf  =  temp 

temp  =  1 
i  =  1 

do  while  i  <=  right 

temp  =  temp  *  i 

i  =  i  +  1 
enddo 
rf  =  temp 

temp  =  1 

i  =  1 

k  =  numasked  -  right 

do  while  i  <=  k 

temp  =  temp  *  i 

i  =  i  +  1 
enddo 
kf  =  temp 

£  =  nf/(kf  *  rf) 

*  EOF  FACTOR. PRG 
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APPENDIX  C 
QUESTION  LISTING 


This  appendix  lists  the  75  questions  contained  in  the 
written  and  computer  final  exams.  The  questions  are  divided 
into  the  three  topic  areas:  introduction,  normalization  and 
management.  Each  topic  area  begins  with  five  critical 
questions  followed  by  twenty  supplementary  questions.  The 
difficulty  level  of  the  question  is  indicated  by  a  number 
from  one  to  three.  This  difficulty  level  was  determined  by 
the  expert  opinion  of  the  instructor  based  on  his  experience 
of  the  questions  in  previous  quarters.  The  meaning  of  those 
numbers  is  as  follows: 

1  less  difficult 

2  average  diffculty 

3  very  difficult 

The  answer  to  each  question  is  indicated  by   >  or 
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INTRODUCTION  QUESTIONS 


Introduction  Critical  Question  1 


Level  3 


Consider   a  STUDENT  file  vith  the  following  fields   used 
in  a  file  processing  system: 


Name 

character 

15 

Rank 

character 

6 

Address 

character 

20 

City 

character 

10 

State 

character 

2 

Zip 

numeric 

5 

Salary 

numeric 

5 

Age 

numeric 

2 

One  of  the  records  in  the  STUDENT  file  contains  the 
following  values: 

Smith  LCDR  202FranklinSt   Monterey  CA  93940  42000  33 


If  the  Postal  authorities  decided  to  increase  the  zip 
code  length  to  six  digits  and  the  application  program  was 
so  modified,  John's  salary  (based  on  the  existing 
data  files)  would: 

a.  not  change  at  all 

b.  would  increase  by  $3 

c.  would  increase  by  $100,003 

d.  would  increase  by  $30,000 
>  e.  would  decrease  by  $21997 

f.  would  decrease  by  $40000 
q.  None  of  the  above  is  true 
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Introduction  Critical  Question  2 


Level  2 


Which  one  of  the  following  portrays  accurately  a   subset 
of  the  functional  components  of  a  DBMS? 


I. 


— > — 

1 
1 

1 
1 
1 

1 
1 
1 
1 
1 

Definition 

Tools 

Subsystem 

1 

1 

l<- 

1 

1 

1 
1 

l<- 
1 

1 

Logical 
I/O  request 

Logical 
I/O  request 

-> 
-> 

1    User 

1     or      1 

DBMS   I 

(Application 
1  Program 

engine  I 

Processing 

Interface 

Subsystem 

II . 


Data 
Design 

1 

l<- 
1 
1 

1 
1 

l<- 
1 

> 

<- 

Logical 

I/O 
requests 

Subsystem 

DBMS 
engine 

Physical 

-->  I/O 
request 

Data 
Administration 

Subsystem 

III. 


1     Logical 
Data     1   I/O  Reguest 

1   DBMS 
->|  engine 

Phys 
I/O  r 

ical 
equest 

UXCC  lQUaty   1  \----   —  —  —  -.-.  —  —  — 

1 

1 

s  1           1 

?  \                    \ 

1   Op   1 

a.  I  and  II  only 

b.  II  and  III  only 

c.  I  and  III  only 

d.  Ill  only 

e.  all  the  above 

1  System  1 

1     DB 

— —  — S 1           1 

> 

I/O 
Comm 

ands 
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Introduction  Critical  Question  3 


Level  3 


The   Prime  Area  of  a  storage  that  uses  index   sequential 
method  contains  the  following  records  (in  unblocked  format): 


Track  1 
Track  2 
Track  3 


REC  5 
RBC  25 
REC  45 


REC  10 
REC  30 
REC  50 


REC  15 
REC  35 
REC  55 


REC  20 
REC  40 
REC  60 


If   REC   22  and  REC  23  were  to  be  added,  the   new  track 
index  would  look  like 


a. 


20   20 

2  40 

3  60 


null 

40   null 

60   null 


1  20   22   address  of  22 

2  40   40   null 

3  60   60   null 


>  c.   1  20  20  null         d.   1    20   22 

2  30  40  address  of  35     2    40   35 

3  60  60  null  3    60   60 

e.   1  20  20  address  of  22 

2  40  40  address  of  33 

3  60  60  null 


address  of  22 
address  of  35 
null 


Introduction  Critical  Question  4 

Consider  the  following  object  diagram 


COMPUTER 


Computer # 

Osver 

Acgdate 

Status 

Station! 

SOFTWARE (MV) 


STUDENT 


Student! 
Sphone 
CLASS (MV) 
APPOINTMENT (MV) 


SOFTWARE 


Sname 
Copy! 

APPOINTMENT (MV) 
COMPUTER (MV) 


Level  1 


CLASS 


I  Cname  I 
I  Tothrs  I 
I  STUDENT (MV)  | 


I 


APPOINTMENT 


Apptdate 

Apptstrt 

CLASS 

STUDENT 

COMPUTER 

SOFTWARE (MV) 
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The  Bachman  diagram  for  the  above  objects  is  given  by: 


I.        I >  COMPUTER  << >>  SOFTWARE  << 


>>  APPOINT  << 
>>   MENT    <- 


I >STUDENT<< >>CLASS< | 

II.       I >  COMPUTER  << >>  SOFTWARE  << I 

I  I 

| >  APPOINT  << I 

| >>   MENT    << | 


I >   STUDENT  << >>  CLASS  < | 

III.      I >  COMPUTER  << >>  SOFTWARE  << 


->  APPOINT  << 
>>   MENT    <- 


I >  STUDENT  << >>  CLASS  < 


a.  only  I  is  true         d.  only  II  and  III  are  true 

b.  only  II  is  true        e.  all  are  true 
>  c.   only  III  is  true       f.  none  are  true 


Introduction  Critical  Question  5  Level  1 

Consider  the  following  table: 

Scheduling       Inventory       Usage  Analysis 

Students  X                                 X 

Class  Rosters  X               X 

Computers  X 

Faculty  X 

Which  of  the  following  is  true? 

I.  Column  I  indicates  the  different  applications 
in  a  database  system  and  the  remaining  columns 
indicate  files  used  by  a  DBMS. 

II.  "X"  indicates  which  file  is  used  by  which  DB 
application. 
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III.  Each  column  in  the  table  is  called  a  "schema". 

IV.  Column  1  indicates  the  different  files  and  the 
remaining  columns  indicte  applications. 

V.  The  table  is  called  a  "schema". 

a.  I,  II,  IV  and  V  only 

>  b.  II,  IV  and  V  only 

c.  II  and  IV  only 

d.  II,  III  and  IV  only 

e.  All  the  above  are  true 
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Introduction  Supplementary  Question  1  Level  2 

The  disadvantages  of  file  processing  systems  are: 

I.  data  is  separated  and  isolated 

II.  data  is  often  duplicated 

III.  application   programs   are   dependent   on   file 
formats 

IV.  processing  is  slower  than  manual  systems 

a.  all  the  above 

>  b.  I,  II  and  III  only 

c.  II ,  III  and  IV  only 

d.  I  and  III  only 

Introduction  Supplementary  Question  2  Level  3 

The  functions  of  a  DBMS  engine  include: 

I.  build  conceptual  schema  structure 

II.  manage  disc  and  memory  space 

III.  provide  object  and  domain  definitions 

IV.  map  logical  I/O  requests  into  physical  activity 

a.  I  and  III 

>  b.  II  and  IV 

c.  I,  II  and  IV 

d.  all  of  the  above 

e.  none  of  the  above 

Introduction  Supplementary  Question  3  Level  3 

Whether  a  property  is  single-  or  multiple-valued  has 
nothing  to  do  with  whether  it  is  an  object  or 
nonobject  property. 

T  or  F 

Introduction  Supplementary  Question  4  Level  2 

Database   data   can  be  updated  in  different   ways   by  . 
user: 

I.  entering  data  via  a  load  utility 

II.  entering  data  with  a  form 

III.  entering  data  in  a  tabular  format 

IV.  entering  data  through  database  views 
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a.  all  of  the  above 

b.  II,  III  and  IV 
>  c.  I,  II  and  III 

d.  I,  II  and  IV 


Introduction  Supplementary  Question  5  Level  2 

Some  of  the  functions  of  a  DBMS  are: 

I.  store,  retrieve  and  update  data 

II.  provide  object  and  Bachman  diagrams 

III.  provide  coordination  and  control  facilities  for 
concurrent  processing 

IV.  provide  facilities  for  backup  and  recovery 

a.  all  the  above 

b.  II  and  IV 

c.  I,  II  and  IV 

>  d.  I,  III  and  IV 

Introduction  Supplementary  Question  6  Level  3 

Integrity  rules  are  very  easy  to  enforce  using  a  DBMS 

T  or  F 

Introduction  Supplementary  Question  7  Level  3 

The  domain  of  an  object  property  is  a  set  of  object 
instances . 

T  or  F 

Introduction  Supplementary  Question  8  Level  3 

The   fact  that  a  property  has  multiple  values  does  not 
imply  that  it  must  be  an  object  property. 

T  or  F 

Introduction  Supplementary  Question  9  Level  2 

An  OBJECT  is  a  named  collection  of  determinants  that 
sufficiently  describes  an  entity  in  the  users  work 
environment . 

T  or  F  < 
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Introduction  Supplementary  Question  10  Level  2 

A  set  of  attributes  with  the  property  that  no  two  tuples 
in  the  relation  can  ever  have  the  same  values  of  these 
attributes  is  called  the  relation  key. 

T  or  F 


Introduction  Supplementary  Question  11  Level  2 

The  Portion  of  an  object  that  is  visible  to  a  particular 
application  is  called  a  VIEW. 

T  or  F 


Introduction  Supplementary  Question  12  Level  2 

A  complex  relationship  between  two  relations  can  be 
represented  by  including  the  key  of  one  relation  as  an 
attribute  in  the  other. 

T  or  F 


Introduction  Supplementary  Question  13  Level  1 

The  components  of  a  database  application  are: 

I .  hardware 

II .  programs 

III.  data 

IV.  procedures 

V.  personnell 

>  a.   all  the  above 

b.  I,  II,  III  and  V 

c.  I,  II  and  III 

d.  Ill,  IV  and  V 

Introduction  Supplementary  Question  14  Level  1 

Some  types  of  database  data  are: 

I.  application  data 

II.  database  metadata 

III.  overhead  data 

IV.  transaction  log  data 
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a.  all  of  the  above 
>  b.  I,  II  and  III 

c.  I  and  III  only 

d.  II  and  IV  only 

Introduction  Supplementary  Question  15  Level  1 

The  different  phases  of  database  system  development  are 

I.  definition 

II.  requirement 

III.  evaluation 

IV.  design 

V.  implement 


a.  I,  II,  III  and  IV 

b.  I,  III,  IV  and  V 

c.  IV  and  V 

>  d.  all  of  the  above 


Introduction  Supplementary  Question  16  Level  1 

Different   attributes   may   take  values   from   the   same 
domain. 

T  or  F 


Introduction  Supplementary  Question  17  Level  1 

Each  record  in  a  relation  is  unique. 

T  or  F 

Introduction  Supplementary  Question  18  Level  1 

A  relation  is  a  flat  file. 
T  or  F 

Introduction  Supplementary  Question  19  Level  1 

To  be  able  to  join  two  relations  there  must  always  be 
one  common  attribute. 

T  or  F 
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Introduction  Supplementary  Question  20  Level  1 

The  SELECT  operation  defines  a  subset  of  tuples  of  a 
given  relation  (i.e.  horizontal  slice)  based  on  a  Boolean 
condition. 

T  or  F 
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NORMALIZATION  QUESTIONS 

Normalization  Critical  Question  1  Level  3 

Consider  the  relation  with  the  following  structure: 

STUDENT_USAGE  (SID,  Compute r_Number,  Class,  Sname,  Total_Hours) 

You  are  also  given:    SID  >>  Computer_Number 

SID   >>  Class 

SID   >  Sname 

SID   >  Total_Hours 

Sname   >  SID 

Which  of  the  following  is  false? 

I .  SID  and  Sname  are  determinants  but  SID  alone  is  the  key 

II.  There  is  at  least  one  transitive  dependency  leading  to 
anomolies 

III.  There  is  exactly  one  MVD 

IV.  The  relation  can  be  upgraded  to  the  next  higher  normal 
form  by  decomposing  it  into  smaller  relations 


>  a.  I,  II  and  III 

b.  II,  III  and  IV 

c.  I,  II  and  IV 

d.  all  the  above 

e.  none  of  the  above 
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Normalization  Critical  Question  2 


Level  3 


Consider  the  following  three  relations  with  the  given 
functional  dependencies.   Identify  the  normal  form  in  which 
each  relation  is  in. 


Rl  (A,  C,  B,  D) 
R2  (A,  B,  E) 
R3  (A,  C,  D,  F) 


a.  Rl  -  2ndNF 
R2  -  3rdNF 
R3  -  2ndNF 

c.  Rl  -  3rdNF 
R2  -  BCNF 
R3  -  2ndNF 

e.  none  of  the  above 


A 

>  B 

AC 

>  D 

A 

>  E 

B 

>  CD 

D 

>  F 

>  b. 

Rl  - 

IstNF 

R2  - 

BCNF 

R3  - 

2ndNF 

d. 

Rl  - 

IstNF 

R2  - 

3rdNF 

R3  - 

2ndNF 

Normalization  Critical  Question  3 


Level  3 


Consider  the  relation  S_J_T  with  three  objects  respectively; 
Student,  Subject  and  Teacher.   The  following  constraints  apply: 

--  for  each  subject,  each  student  of  that  subject  is 
taught  by  only  one  teacher 

--  each  teacher  teaches  only  one  subject 

—  each  subject  is  taught  by  several  teachers 

Which  of  the  following  is  true? 

I.  S_J_T  is  in  BCNF 

II .  S_J_T  has  MVDs 

III.  S_J_T  has  one  possible  composite  key 


a.  I  only 

b.  II  only 

c.  Ill  only 


d.  all  of  the  above 
>  e.  none  of  the  above 
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Normalization  Critical  Question  4  Level  1 

When  these  two  relations  are  joined, 


PART 

p# 

COLOR 

CITY 

PI 

red 

LA 

P2 

blue 

NY 

P3 

green 

SF 

P4 

blue 

NY 

SUPPLIER 

SI 

P# 

SI 

PI 

SI 

P2 

S2 

PI 

Which  one  of  the  following  is  true? 

a.  there  will  be  exactly  12  tuples  in  the  resultant  join 

>  b.  there  will  be  exactly   3  tuples  in  the  resultant  join 

c.  there  will  be  exactly   4  tuples  in  the  resultant  join 

d.  there  will  be  exactly   7  tuples  in  the  resultant  join 

e.  none  of  the  above 


Normalization  Critical  Question  5  Level  1 

Given  the  relations  below: 

CABLE  CUSTOMER  CONVERTOR 


I  I  III  I 

Figure  1   I  Account*  I — 0 0--|  serial#  I...  I  Account*  I 

I I  I I I I 

SHIPMENT  PARCEL 


I  I  I  I     I  I 

Figure  2   I  Container*  I  —  | — 0 —  I  Parcel*  I...  I  Container  I 

I I        \l I I I 

PART  BIN  LOCATION 


I  III  III 

Figure    3  I    Part*    I...  I    Bin*    I  — O |--|    Bin*    I... I 

I I I I  I I I 


PLAYER  POSITION 


III  I  I 

Figure  4        I  Name  |...|--| I  —  I  Position  I 

I I 1/  \l I 
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Which  of  the  following  is  true? 

I.  Fig  1  represents  an  optional-optional  relationship 

II.  Fig  2  represents  a  mandatory-optional  relationship 

III.  Fig  3  represents  an  optional-mandatory  relationship 

IV.  Fig  4  represents  a  mandatory-mandatory  relationship 


a.  I  and  IV  only 

b.  II  and  III  only 

c.  I,  II  and  III  only 

d.  I,  II  and  IV  only 

e.  all  the  above  are  true 
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Normalization  Supplemental  Question  1  Level  3 

A  loss  projection  occurs  when 

>  a.   false  data  is  generated  by  a  projection 

b.  a  projection  loses  attribute  values 

c.  a  projection  loses  tuples 

d.  a   projection   cannot  be   made   on   two   specific 
relations 

Normalization  Supplemental  Question  2  Level  3 

Which   one   of   the   following  is  true   in   one   to   one 
relationships? 

I.  Attributes  that  have  a  one  to  one  relationship  must 
occur  together  in  at  least  one  relation.  (call  the 
relation  R  and  the  attributes  A  and  B) 

II.  Both  A  and  B  must  be  part  of  the  key  of  R. 

III.  An  attribute  can  be  added  to  R  if  it  is  functionally 
determined  by  A  or  B. 

IV.  A  and  B  must  occur  together  in  R,  but  should  not  occur 
together  in  other  relations. 

a.  I,  II  and  III 

>  b.  I,  III  and  IV 

c.  II  and  III 

d.  all  of  the  above 

Normalization  Supplemental  Question  3  Level  3 

When  evaluating  database  design 

a.  machine  efficiency  has  highest  priority 

b.  elimination  of  modification  anomalies  has   the 
highest  priority 

c.  relation  independence  has  the  highest  priority 
>  d.   priorities   will  be   different   depending   upon 

requirements 

Normalization  Supplemental  Question  4  Level  2 

Primary   keys   must   be  unique   within   a   relation   but 
candidate  keys  do  not  have  to  be  unique  within  a  relation. 

T  or  F  < 


70 


Normalization  Supplemental  Question  5  Level  3 

In  the  following  functional  dependancy  A  ->  B  ->  C, 

a.  the  dependancy  is  a  multivalued  dependancy 

>  b.   C  is  a  transitive  dependancy  with  A  as  its 

determinant 

c.  C  is  the  determinant 

d.  A  and  C  are  not  dependent 

Normalization  Supplemental  Question  6  Level  2 

If  a  relation  is  in  domain  key  normal  form,  it 

>  a.  could  have  no  modification  anomalies 

b.  may  still  have  transitive  dependencies 

c.  could  still  have  join  dependencies 

d.  could  still  have  multivalued  dependencies 

Normalization  Supplemental  Question  7  Level  3 

A  relation  in  first  normal  form  can  generally  be   placed 
in  second  normal  form  by 

>  a.   projections 

b.  joins 

c.  selections 

d.  products 

Normalization  Supplemental  Question  8  Level  3 

An  interrelation  constraint  exists  when 

>  a.   two  or  more  relations  share  the  same  attribute 

b.  two  attributes  in  the  relation  have  the  same 
determinant 

c.  two  determinants  in  a  relation  have   the   same 

subject  attribute 

d.  two  or  more  attributes  of  a  candidate  key  have 
different  determinants 

Normalization  Supplementary  Question  9  Level  3 

A  modification  anomaly  occurs  when  change  to  one  entity 
causes  a  change  to  a  second  entity. 

T  or  F  < 
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Normalization  Supplemental  Question  10  Level  3 

Multivalued  dependencies  occur  when 

a.  several  attributes  are  dependent   upon   the 
same  determinant 

b.  a  single  attribute  has  several  determinants 

c.  there  are  several  transitive  dependencies 

>  d.   several  independent  attributes  are  dependent 

on  one  other  attribute 

Normalization  Supplemental  Question  11  Level  3 

Which  of  the  following  is  not  a  relational  design 
evaluation  criterion: 

a.  elimination  of  modification  anomolies 

b.  relation  independence 

>  c.  ease  of  use 

d.  reduction  of  key  attributes 

Normalization  Supplemental  Question  12  Level  3 

Attributes  that  have  a  many  to  many  relationship  can 
exist  together  in  a  single  relation  (although  this  would 
create  redundancies).  Assume  two  such  attributes  E  and  F, 
residing  together  in  a  relation  T. 

Which  of  the  following  is  false? 

I .  The  key  of  T  must  be  E 

II.  The  key  of  T  must  be  F 

III.  Both  E  and  F  can  be  candidate  keys 

IV.  An  attribute  can  be  added  to  T  if  it  is   determined 
by  E  or  F 

a.  I  only 

b.  II  only 

c.  Ill  and  IV  only 
>  d.  all  the  above 

e.  none  of  the  above 
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Normalization  Supplemental  Question  13  Level  1 

A  relation  Is  In  domain/key  normal  form  when 

a.  the  domain  of  the  key  Is  the  function  of  the   key 

b.  the  key  is  the  function  of  the  domain 

c.  it  has  no  multivalued  dependencies 

>  d.   every  constraint  is  a  logical  consequence   of 
the  definition  of  keys  and  domains 

Normalization  Supplemental  Question  14  Level  1 

If   a   relation  is  in  third  normal  form,  it  must   be   in 
second  normal  form. 

T  or  F 

Normalization  Supplemental  Question  15  Level  1 

A  relation  is  in  Boyce-Codd  normal  form  if 

a.  it  has  only  one  key 

b.  it  has  no  transitive  dependencies 

>  c.   every  determinant  is  a  candidate  key 
d.   there  are  no  deletion  anomalies 

Normalization  Supplemental  Question  16  Level  1 

If  a  tuple  cannot  be  inserted  into  a  relation  until  an 
additional   fact   about   another   entity   is   known,   it   is 
referred  to  as: 

a.  a  deletion  anomaly 

>  b.  an  insertion  anomoly 

c.  a  functional  dependency 

d.  an  insertion  redundancy 

Normalization  Supplemental  Question  17  Level  1 

If   a   relation  meets  the   criterion   that   all   nonkey 
attributes  are  dependent  upon  all  of  the  key,  it  must  be  in: 

>  a.  second  normal  form 

b.  fourth  normal  form 

c.  domain/key  nornal  form 

d.  Boyce-Codd  normal  form 
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Normalization  Supplemental  Question  18  Level  1 

In  the  following  functional  dependency  A  ->  B  ->  C,  the 
determinant (s ) : 

a.  is  A  only 

b.  is  C  only 

>  c.   are  A  and  B 
d.   are  B  and  C 


Normalization  Supplemental  Question  19  Level  1 

Multivalued  dependencies  always  exist  in  pairs. 

T  or  F 

Normalization  Supplemental  Question  20  Level  1 

If  an  attribute  is  a  key  it  must  be  a  determinant. 

T  or  F 
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MANAGEMENT  QUESTIONS 


Management  Critical  Question  1 


Level  3 


You  are   given  this  transaction  log  at  the   time   the 
database  crashed. 


Relative 

Record 

Number 

1    OT1 

0 

2 

11: 

:42 

START 

2    OT1 

1 

4 

11: 

43 

MODIFY 

CUST100 

3    OT2 

0 

8 

11: 

.46 

START 

4    OT1 

2 

5 

11, 

47 

MODIFY 

SP  AA 

5    OT1 

4 

7 

11 

:47 

INSERT 

ORDER  1 

6    CT1 

0 

9 

11: 

48 

START 

7    OT1 

5 

0 

11' 

:49 

COMMIT 

8    OT2 

2 

0 

11 

:50 

COMMIT 

9    CT1 

6 

10 

11 

:51 

MODIFY 

SP  AA 

10   CT1 

9 

0 

11: 

51 

COMMIT 

(old  value)  (new  value) 


(old  value) 


(new  value) 
(value ) 


(old  value)  (new  value) 


Which  of  the  transactions  will  be  reprocessed  after  the 
last  save? 


I  . 

II. 

Ill 


OT1 
OT2 
CT1 


a.  I  only 

b.  II  only 

c.  Ill  only 

d.  all  of  the  above 
>  e.  none  of  the  above 
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Management  Critical  Question  2 


Level  2 


There  are  three  users  in  a  multiuser  DB.   Each  wants  to 
perform  the  following  sequence  of  events. 


USER  A 


USER  B 


Read  rec  100 
Change  rec  100 
Write  rec  100 


Read  rec  200 
Change  rec  200 
Write  rec  200 


USER  C 


Read  rec  200 
Change  rec  200 
Write  rec  200 


Which  of  the  following  sequences  violates  serializability? 


I. 
Read  100  for  A 
Read  200  for  B 
Change  100  for  A 
Write  100  for  A 
Read  200  for  C 
Change  200  for  B 
Write  200  for  B 
Change  200  for  C 
Write  200  for  C 


II. 
Read  200  for  B 
Read  100  for  A 
Change  100  for  A 
Write  100  for  A 
Change  200  for  B 
Read  200  for  C 
Change  200  for  C 
Write  200  for  C 
Write  200  for  B 


III. 
Read  200  for  C 
Change  200  for  C 
Read  100  for  A 
Write  200  for  C 
Change  100  for  A 
Read  200  for  B 
Change  200  for  B 
Write  100  for  A 
Write  200  for  B 


Select  your  answer: 


a.  I  and  II  only 

b.  II  and  III  only 

c.  I  and  III  only 

d.  Ill  only 
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Management  Critical  Question  3  Level  3 

Five  records  are  stored  physically  on  a  disc  track  as 
shown  below: 

RRN  4 


/  \ 

RRN  5   /  \  RRN  3 

/  \ 

_  /  \  _ 

I  (a  very  I 

I      crude  disc  track)       I  RRN  2 

I  I 

I  l_ 

\  / 

\  / 

\  /   RRN  1 

\ / 


The  actual  contents  of  each  record  is  given  in  the  following 
table : 


RPN 

SUPNAME 

PARTID 

PRICE 

1 

Topnotch 

P003 

9.95 

2 

Ugo 

P014 

0.99 

3 

Bestparts 

P020 

29.95 

4 

Acme 

P008 

99.99 

5 

Gotcha 

P002 

11.00 

Assume  you  access  the  data  in  the  following  ways: 

I.  Access  the  entire  file  in  the  order  specified  by  a 
linked  list  created  according  to  ascending  order 
(alphabetic)  by  Supname . 

II.  Access  the  entire  file  in  the  order  specified  by  a 
linked  list  created  according  to  ascending  order 
(numeric)  by  PartlD. 

Assuming  the  disc  rotates  in  a  clockwise  direction, 
which  of  the  following  is  true? 

>  a.  There  will  be  no  difference  in  the   number   of  disc 
rotations   if  the  entire  file  were  to  be  accessed   by 

either  I  or  II. 

b.  Method  I  will  be  faster  by  one  rotation  than  method  II. 

c.  Method  II  will  be  faster  by  one  rotation  than  method  I. 

d.  Method  I  will  be  faster  by  two  rotations  than  method  II 

e.  Method  II  will  be  faster  by  two  rotations  than  method  I 
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Management  Critical  Question  4  Level  2 

The  following  table  is  an  example  of 

PATIENT  Record 

Subjects  Who  Know  Subjects  Who  Know 

Password  SICK  Password  PAIN 

Read  Y  Y 

Insert  Y  N 

Actions:    Modify  Y  N 

Delete  Y  N 

Grant  N  N 


a.  Fernandez,  Summers,  Wood  model  of  database  security 

b.  subject-orientated  security 
>  c.  entity-orientated  security 

d.  all  of  the  above 
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Management  Critical  Question  5 


Level  1 


You  are  given  the  following  table: 


Relative 

Record 

Student- 

Class- 

Number 

number 

number 

Semester 

1 

200 

70 

88S 

2 

100 

30 

89F 

3 

300 

20 

89F 

4 

200 

30 

88S 

5 

300 

70 

88S 

6 

100 

20 

88S 

Determine  which 

one  is  the 

correct  inver 

ted 

list  ( 

(or  index) 


I . 


II. 


Ill . 


Relative 

Relative 

Relative 

Student- 

Record 

Class- 

Record 

Record 

number 

Number 

number 

Number 

Semeste 

r     Number 

100 

2 

20 

3 

88S 

1 

100 

6 

20 

6 

88S 

4 

200 

1 

30 

2 

88S 

5 

200 

4 

30 

4 

88S 

6 

300 

3 

70 

1 

89F 

2 

300 

5 

70 

5 

89F 

3 

a.  I  only 

b.  II 

only 

c.  Ill 

only 

d.  I  a 

nd  II  only 

>  e.  all 

the  above 
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Management  Supplemental  Question  1  Level  1 

A  binary  relationship  involves  at  least  two  record 
types . 

T  or  F 


Management  Supplemental  Question  2  Level  3 

When  the  relationship  is  a  l:n  and  is  mandatory  in  both 
directions,   there  is  a  strong  likelyhood  that   the   records 
are  describing  different  aspects  of  the  same  object. 

T  or  F 


Management  Supplemental  Question  3  Level  3 

An  object  relationship  is  one-to-one  if: 

I.  object  A  contains  object  B  as  a  single-valued 
object  property 

II.  object  B  contains  Object  A  as  a  single- 
valued  object  property 

III.  object  B  does  not  contain  object  A 


a.  only  condition  I  is  necessary 

>  b.  conditions  I  and  II  alone  are  sufficient 

c.  conditions  II  or  III  alone  are  sufficient 

d.  conditions  I,  II  and  III  are  all  necessary 


Management  Supplemental  Question  4  Level  3 

If   A  is  related  to  B  and  B  is  not  related  to  A,   it   is 
referred  to  as  0:1  relationship. 

T  or  F 


Management  Supplemental  Question  5  Level  3 

The  multi-valued  attributes  in  a  composite  object  never 
represent  another  object. 

T  or  F 


Management  Supplemental  Question  6  Level  3 
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The  keys  of  two  relations  having  a  compound-object 
relationship  between  them  do  not  have  a  common  attribute. 


T  or  F 

Management  Supplemental  Question  7  Level  1 

An  atomic  transaction  is 

a.  any  set  of  activities  that  operate  on  the  same 
data-items 

>  b.  a  set  of  activities  that  form  one  logical  unit  of 

work 

c.  any  set  of  activities  that  can  be  performed  with 
one  instruction  to  the  DBMS 

d.  any  set  of  activities  that  can  be  performed 
concurrently 

Management  Supplemental  Question  8  Level  2 

To  roll  forward, 

>  a.  restore  the  database  from  a  save  and  apply  after 

images 

b.  use  the  current  copy  of  the  database   and   apply 
the  after  images 

c.  restore  the  database  and  apply  before  images 

d.  use  the  current  copy  of  the  database  and  apply 
before  images 

Management  Supplemental  Question  9  Level  3 

A  check  point  record  is  written 

a.  after  all  updates  have  been  made  to  the  database 

b.  after   all   requests  have  been  written   to   the 
transation  log 

>  c.  after   all   requests  have  been  written   to   the 

database  and  the  transaction  log 
d.  before  updates  have  been  written  to  the  database 


81 


Management  Supplemental  Question  10  Level  1 

In  the  Fernandez,  Summers,  and  Wood  Model  of  database 
security,  a  subject  can  be 

>  a.  the  program  that  will  be  processing  the  database 

b.  the  element  of  the  database  that  will  be  protected 

c.  the  processing  permissions  given  to  a  user 

d.  the  user's  password 


Management  Supplemental  Question  11  Level  2 

Constraints  on  user's  activity  can  be  enforced  by 

a.  subject-oriented  security 

b.  object  oriented  security 

c.  password  matrices 
>  d.  user  exits 

Management  Supplemental  Question  12  Level  1 

Functions  of  a  DBA  include: 

I.  Management  of  data  activity 

II.  Management  of  database  structure 

III.  Database  Personnel  and  Placement 

IV.  Management  of  the  DBMS 

a.  I,  II,  IV  only 

b.  II  and  IV  only 

>  c.   All  of  the  above 
d.   IV  only 

Management  Supplemental  Question  13  Level  1 

Concurrent  processing  implies  that  the  DBMS  processes 
the  queries  of  various  users  simultaneously. 

T  or  F 

Management  Supplemental  Question  14  Level  2 

Ser ializability  ensures  that  results  of  two  concurrent 
transactions  are  the  same  as  they  would  have  been  had  they 
been  processed  one  at  a  time. 

>   T  or  F 
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Management  Supplemental  Question  15  Level  1 

Database  changes  that  have  been  made  by  a  transaction  In 
process  are  said  to  be  uncommitted. 

T  or  F 


Management  Supplemental  Question  16  Level  1 

Concurrent  processing  does  not  lead  to  lost  updates  If 
transactions  are  only  reading  data. 

T  or  F 


Management  Supplemental  Question  17  Level  1 

Deadlock  occurs  when  each  of  two  transactions  is  waiting 
for  a  resource  the  other  has  locked. 

T  or  F 


Management  Supplemental  Question  18  Level  1 

Checkpoints   are   snapshots   of   updated   parts   of    a 
database . 

T  or  F 


Management  Supplemental  Question  19  Level  1 

The   Performance   Monitor   maintains   the   configuration 
control  of  the  DBMS. 

T  or  F 


Management  Supplemental  Question  20  Level  1 

Interleaving  means  that  while  two  or  more  transactions 
are  processing  concurrently  the  computer  system  executes 
some  instructions  from  one,  then  executes  some  from  the 
other,  switching  back  and  forth  between  them. 

T  or  F 
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APPENDIX  D 
COMPUTER-AIDED  TEST  QUESTIONNAIRE 


LAST  NAME LAST  FOUR  DIGITS  OF  SSN 


1.     What  are  your  general  feelings  toward   this   computer 
aided  testing  procedure? 

very      somewhat  somewhat       very 

negative     negative     indifferent     positive     positive 


2.     How   do  you  feel  that  not  being  able  to   explain   your 

answers  affects  the  accuracy  of  the  test? 

very      somewhat  somewhat      very 

negatively  negatively   indifferently  positively  positively 


3.     How   do   you  feel  about  not   having   to   answer   essay 
questions? 

very     somewhat  somewhat     very 

negatively  negatively   indifferently   positively  positively 


4.    Did  you  feel  more  pressure  with  the  computer  test   than 
you  have  usually  felt  during  other  written  exams? 

Yes  or  No 


5.    Rate  your  comfort  level  in  using  computers. 

very  indifferent  very 

uncomfortable  comfortable 

somewhat  somewhat 

uncomfortable  comfortable 


6.    Do  you  think  a  computer  aided  exam  tests  the   knowledge 
of  the  student  more  effectively  than  a  written  exam 

Yes  or  No 
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7.    Do  you  think  that  making  such  programs  available   for 
practice   would   help  students   learn   subject   matter 
better? 

Yes  or  No 


8.  Please  note  any  comments,  likes/dislikes  or 
recommendations  you  have  about  this  test  or  the 
concept  of  computer  aided  testing  in  general? 
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APPENDIX  E 
EXPERIMENT  DATA 


STUDENT 

SECTION 

EXAM1 

EXAM2 

FINAL 

COMPSCOR 

1 

1 

40 

43 

56 

2.34 

2 

1 

49 

47 

76 

4 

3 

1 

48 

43 

59 

3.34 

4 

1 

46 

46 

46 

2 

5 

1 

50 

42 

51 

2 

6 

1 

46 

41 

59 

3 

7 

1 

49 

45 

64 

3.34 

8 

1 

39 

41 

64 

2.67 

9 

1 

42 

48 

74 

4 

10 

1 

47 

47 

60 

2.34 

11 

1 

45 

45 

74 

3 

12 

1 

44 

37 

45 

2 

13 

1 

48 

44 

70 

3 

14 

1 

48 

47 

67 

3.34 

15 

1 

45 

46 

58 

2 

16 

1 

47 

45 

74 

3.67 

17 

1 

49 

43 

68 

4 

18 

1 

46 

40 

52 

2 

19 

1 

43 

43 

38 

1 

20 

1 

33 

45 

55 

1.67 

21 

1 

34 

38 

45 

1.67 

22 

2 

47 

48 

72 

3.34 

23 

2 

43 

42 

57 

2.34 

24 

2 

48 

49 

65 

4.34 

25 

2 

44 

46 

75 

3.34 

26 

2 

46 

43 

60 

3 

27 

2 

39 

45 

55 

3.34 

28 

2 

44 

44 

61 

3.67 

29 

2 

43 

46 

51 

2.34 

30 

2 

48 

48 

72 

3.34 

31 

2 

49 

46 

75 

3.34 

32 

2 

43 

41 

68 

3.34 

33 

2 

38 

43 

51 

2.34 

34 

2 

39 

43 

48 

1.34 
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STUDENT   QUEST1   QUEST2   QUEST3   QUEST4    QUESTS   QUEST6 

GENLATT  XPLNANS  NOESSYQ  PCVDPRES  COMFLVL  PCVDEFCT 


1 

0 

-2 

0 

1 

2 

1 

2 

0 

0 

2 

1 

4 

0 

3 

-2 

-2 

-4 

0 

4 

0 

4 

2 

4 

4 

1 

4 

0 

5 

-2 

-4 

-4 

0 

4 

0 

6 

2 

0 

-2 

1 

2 

0 

7 

2 

-2 

0 

0 

2 

0 

8 

-2 

0 

-2 

0 

-2 

0 

9 

0 

-2 

2 

1 

4 

0 

10 

0 

0 

2 

0 

2 

11 

4 

2 

2 

1 

0 

0 

12 

2 

-4 

4 

1 

2 

1 

13 

2 

-2 

-2 

0 

4 

0 

14 

2 

0 

4 

1 

4 

0 

15 

2 

2 

2 

1 

2 

0 

16 

4 

-2 

-2 

4 

1 

17 

4 

2 

4 

1 

4 

1 

18 

2 

0 

4 

1 

0 

0 

19 

-4 

0 

0 

-2 

0 

0 

20 

0 

-2 

0 

1 

2 

0 

21 

2 

2 

4 

1 

-4 

0 

22 

4 

-2 

4 

1 

4 

1 

23 

24 

0 

-2 

2 

1 

4 

0 

25 

-4 

-2 

4 

0 

4 

0 

26 

-4 

-4 

0 

0 

4 

1 

27 

0 

-2 

4 

0 

4 

0 

28 

-2 

2 

2 

0 

-2 

0 

29 

-2 

-2 

0 

0 

-2 

0 

30 

2 

-2 

4 

0 

4 

0 

31 

-2 

-2 

0 

0 

-2 

0 

32 

-2 

0 

0 

0 

4 

1 

33 

34 

2 

2 

4 

0 

4 

0 
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UDENT 

QUEST7 

CTIME 

WTIME 

NUMQUEST 

(MIN) 

(MIN) 

1 

1 

46 

120 

29 

2 

1 

28 

120 

45 

3 

1 

12 

120 

17 

4 

1 

27 

120 

18 

5 

0 

18 

75 

19 

6 

1 

32 

107 

25 

7 

1 

35 

119 

45 

8 

0 

19 

103 

21 

9 

1 

15 

120 

33 

10 

1 

40 

120 

26 

11 

1 

55 

120 

60 

12 

1 

25 

114 

41 

13 

1 

26 

117 

35 

14 

1 

29 

104 

23 

15 

1 

22 

120 

40 

16 

0 

41 

120 

46 

17 

1 

25 

90 

38 

18 

1 

19 

85 

20 

19 

0 

17 

120 

9 

20 

1 

45 

120 

27 

21 

1 

56 

120 

26 

22 

1 

63 

100 

17 

23 

26 

80 

9 

24 

1 

53 

70 

25 

25 

0 

48 

85 

52 

26 

0 

70 

113 

60 

27 

1 

44 

95 

20 

28 

1 

61 

94 

29 

29 

1 

77 

53 

27 

30 

1 

56 

78 

24 

31 

0 

67 

112 

36 

32 

1 

50 

70 

17 

33 

44 

60 

9 

34 

1 

27 

70 

20 
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